Feature selection with kernelized multi-class support vector machine

• Published in: Pattern recognition Vol. 117; p. 107988
• Main Authors: Guo, Yinan, Zhang, Zirui, Tang, Fengzhen
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.09.2021
• Subjects: Feature selection; Kernel machine; Multi-class support vector machine; Recursive feature elimination; Feature selection; Multi-class support vector machine; Recursive feature elimination; Kernel machine
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2021.107988

•We propose a new nonlinear feature selection method based on kernelized multiclass support vector machine (COMSVM).•The recursive feature elimination algorithm is improved by adding the batch elimination and the rescreening process.•The fast recursive feature elimination algorithm is implemented with the proposed COMSVM.•The experiment results demonstrate the efficiency of our new method. Feature selection is an important procedure in machine learning because it can reduce the complexity of the final learning model and simplify the interpretation. In this paper, we propose a novel non-linear feature selection method that targets multi-class classification problems in the framework of support vector machines. The proposed method is achieved using a kernelized multi-class support vector machine with a fast version of recursive feature elimination. The proposed method selects features that work well for all classes, as the involved classifier simultaneously constructs multiple decision functions that separates each class from the others. We formulate the classifier as a large optimisation problem, and iteratively solve one decision function at a time, leading to a lower computational time complexity than when solving the large optimisation problem directly. The coefficients of the classifier are then used as a ranking criterion in the accelerated recursive feature elimination by adding batch elimination and a rechecking process. Experimental results on several datasets demonstrate the superior performance of the proposed feature selection method.