Optimal feature selection using information maximization: Case of biomedical data

The hybrid information maximization (HIM) algorithm is derived. This algorithm is based on maximizing the mutual information (MI) between the input and output of a network using the infomax principle, and between outputs of different network modules using the Imax algorithm. These two folds enable reducing the redundancy in output units in addition to selecting higher order features from input units. In this paper, we analyze the proposed algorithm and generalize the learning procedure of the Imax algorithm. We show that the proposed HIM algorithm provides a better representation of input compared to the original two algorithms when used separately. An example showing the power of the HIM algorithm in the analysis of EEG data is discussed.