Feature Selection with Kernel Class Separability

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 30; no. 9; pp. 1534 - 1546
• Main Author: Wang, Lei
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.09.2008; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Application software; Applied sciences; Artificial Intelligence; Classification; Computer science; control theory; systems; Criteria; Data processing. List processing. Character string processing; Exact sciences and technology; Feature evaluation and selection; Feature extraction; Image Enhancement - methods; Image Interpretation, Computer-Assisted - methods; Kernel; Kernels; Maximization; Memory organisation. Data processing; Numerical stability; Optimization; Parameter estimation; Pattern analysis; Pattern classification; Pattern Recognition, Automated - methods; Robustness; Software; Strategy; Support vector machine classification; Support vector machines; Tuning; Separability; Discriminant analysis; Search strategy; Statistical analysis; Kernel Fisher Discriminant Analysis; Support Vector Machines; Optimization; Fisher information; Kernel method; Kernel class separability; Alignment; Multidimensional analysis; Model matching; Pattern classification; Vector support machine; Selection criterion; Pattern analysis; Artificial intelligence; Numerical stability; feature selection
• ISSN: 0162-8828; 1939-3539
• DOI: 10.1109/TPAMI.2007.70799

Classification can often benefit from efficient feature selection. However, the presence of linearly nonseparable data, quick response requirement, small sample problem and noisy features makes the feature selection quite challenging. In this work, a class separability criterion is developed in a high-dimensional kernel space, and feature selection is performed by the maximization of this criterion. To make this feature selection approach work, the issues of automatic kernel parameter tuning, the numerical stability, and the regularization for multi-parameter optimization are addressed. Theoretical analysis uncovers the relationship of this criterion to the radius-margin bound of the SVMs, the KFDA, and the kernel alignment criterion, providing more insight on using this criterion for feature selection. This criterion is applied to a variety of selection modes with different search strategies. Extensive experimental study demonstrates its efficiency in delivering fast and robust feature selection.