Estimating redundancy information of selected features in multi-dimensional pattern classification

• Published in: Pattern recognition letters Vol. 32; no. 4; pp. 590 - 596
• Main Authors: Jung, Chi-Sang, Seo, Hyunson, Kang, Hong-Goo
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 01.03.2011; Elsevier
• Subjects: Applied sciences; Conditional mutual information; Exact sciences and technology; Feature selection; Information, signal and communications theory; Miscellaneous; Pattern recognition; Redundancy information; Signal processing; Speech/music discrimination; Telecommunications and information theory; Redundancy information; Speech/music discrimination; Feature selection; Conditional mutual information; Performance evaluation; Dynamic response; Redundancy; Pattern recognition; Algorithm; Implementation; Accuracy; Discrimination; Pattern classification; Signal processing; Feature extraction; Mutual information
• ISSN: 0167-8655; 1872-7344
• DOI: 10.1016/j.patrec.2010.11.023

► Redundant information between features should be considered for feature selection. ► Estimating conditional mutual information between selected and candidate features. ► The proposed method accurately estimated the amount of information. ► The proposed algorithm had higher performance than conventional algorithms. This paper proposes a novel criterion for estimating the redundancy information of selected feature sets in multi-dimensional pattern classification. An appropriate feature selection process typically maximizes the relevancy of features to each class and minimizes the redundancy of features between selected features. Unlike to the relevancy information that can be measured by mutual information, however, it is difficult to estimate the redundancy information because its dynamic range is varied by the characteristics of features and classes. By utilizing the conceptual diagram of the relationship between candidate features, selected features, and class variables, this paper proposes a new criterion to accurately compute the amount of redundancy. Specifically, the redundancy term is estimated by conditional mutual information between selected and candidate features to each class variable, which does not need a cumbersome normalization process as the conventional algorithm does. The proposed algorithm is implemented into a speech/music discrimination system to evaluate classification performance. Experimental results by varying the number of selected features verify that the proposed method shows higher classification accuracy than conventional algorithms.