A hybrid information maximisation (HIM) algorithm for optimal feature selection from multi-channel data

A novel feature selection algorithm is derived for multi-channel data. This algorithm is a hybrid information maximisation (HIM) technique based on (1) maximising the mutual information between the input and output of a network using the infomax algorithm proposed by Linsker (1988), and (2) maximising the mutual information between outputs of different network modules using the Imax algorithm introduced by Becker (see Network Computation in Neural Systems, vol.7, p.7-31, 1996). The infomax algorithm is useful in reducing the redundancy in the output units, while the Imax algorithm is capable of selecting higher order features from the input units. In this paper, we analyse the two methods and generalise the learning procedure of the Imax algorithm to make it suitable for maximising the mutual information between multi-dimensional output units from different network modules contrary to the original Imax algorithm which only maximises mutual information between two output units. We show that the proposed HIM algorithm provides a better representation of the input compared to the original two algorithms when used separately. Finally, the HIM is evaluated with respect to biological plausibility in the case of feature selection from two-channel EEG data.