Discriminative Least Squares Regression for Multiclass Classification and Feature Selection

• Published in: IEEE transaction on neural networks and learning systems Vol. 23; no. 11; pp. 1738 - 1754
• Main Authors: Xiang, Shiming, Nie, Feiping, Meng, Gaofeng, Pan, Chunhong, Zhang, Changshui
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.11.2012; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Artificial intelligence; Classification; Computer science; control theory; systems; Data processing. List processing. Character string processing; Exact sciences and technology; Expansion; Feature selection; Indexes; Learning; Learning and adaptive systems; Least squares method; least squares regression; Machine learning; Machine learning algorithms; Memory organisation. Data processing; multiclass classification; Neural networks; Norms; Optimization; Prediction algorithms; Regression; Software; sparse learning; Studies; Training; Trains; Vectors; Feature selection; multiclass classification; Least squares method; Sparse matrix; L2 approximation; least squares regression; sparse learning; Artificial intelligence; Modeling; Multiple classification
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2012.2212721

This paper presents a framework of discriminative least squares regression (LSR) for multiclass classification and feature selection. The core idea is to enlarge the distance between different classes under the conceptual framework of LSR. First, a technique called ε-dragging is introduced to force the regression targets of different classes moving along opposite directions such that the distances between classes can be enlarged. Then, the ε-draggings are integrated into the LSR model for multiclass classification. Our learning framework, referred to as discriminative LSR, has a compact model form, where there is no need to train two-class machines that are independent of each other. With its compact form, this model can be naturally extended for feature selection. This goal is achieved in terms of L 2,1 norm of matrix, generating a sparse learning model for feature selection. The model for multiclass classification and its extension for feature selection are finally solved elegantly and efficiently. Experimental evaluation over a range of benchmark datasets indicates the validity of our method.