Feature selection in mixed data: A method using a novel fuzzy rough set-based information entropy

• Published in: Pattern recognition Vol. 56; pp. 1 - 15
• Main Authors: Zhang, Xiao, Mei, Changlin, Chen, Degang, Li, Jinhai
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.08.2016
• Subjects: Entropy; Entropy (Information); Equivalence; Feature selection; Fuzzy rough set theory; Fuzzy set theory; Fuzzy sets; Information entropy; Mathematical models; Mixed data; Set theory; Similarity; Feature selection; Fuzzy rough set theory; Information entropy; Mixed data
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2016.02.013

Feature selection in the data with different types of feature values, i.e., the heterogeneous or mixed data, is especially of practical importance because such types of data sets widely exist in real world. The key issue for feature selection in mixed data is how to properly deal with different types of the features or attributes in the data set. Motivated by the fuzzy rough set theory which allows different fuzzy relations to be defined for different types of attributes to measure the similarity between objects and in view of the effectiveness of entropy to measure information uncertainty, we propose in this paper a fuzzy rough set-based information entropy for feature selection in a mixed data set. It is proved that the newly-defined entropy meets the common requirement of monotonicity and can equivalently characterize the existing attribute reductions in the fuzzy rough set theory. Then, a feature selection algorithm is formulated based on the proposed entropy and a filter-wrapper method is suggested to select the best feature subset in terms of classification accuracy. An extensive numerical experiment is further conducted to assess the performance of the feature selection method and the results are satisfactory. •A novel fuzzy rough set-based information entropy is constructed for mixed data.•The proposed entropy can equivalently characterize the existing attribute reductions in the fuzzy rough set theory.•A feature selection algorithm is formulated based on the proposed entropy.•A filter-wrapper method is suggested to select a best feature subset.