Sliced Inverse Regression With Adaptive Spectral Sparsity for Dimension Reduction

• Published in: IEEE transactions on cybernetics Vol. 47; no. 3; pp. 759 - 771
• Main Authors: Xu, Xiao-Lin, Ren, Chuan-Xian, Wu, Ran-Chao, Yan, Hong
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.03.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation models; Algorithms; Computational modeling; Covariance matrices; Dimension reduction; Input variables; Machine learning; Pattern analysis; Pattern classification; Principal component analysis; Reduction; Regression analysis; Regression coefficients; Regression models; robust classification; Robustness; Robustness (mathematics); sliced inverse regression (SIR); sparse model; Sparsity; spectral regression; Vectors (mathematics)
• ISSN: 2168-2267; 2168-2275; 2168-2275
• DOI: 10.1109/TCYB.2016.2526630

Dimension reduction is an important topic in pattern analysis and machine learning, and it has wide applications in feature representation and pattern classification. In the past two decades, sliced inverse regression (SIR) has attracted much research efforts due to its effectiveness and efficacy in dimension reduction. However, two drawbacks limit further applications of SIR. First, the computation complexity of SIR is usually high in the situation of high-dimensional data. Second, sparsity of projection subspace is not well mined for improving the feature selection and model interpretation abilities. This paper proposes to compute the SIR projection vectors in the spectral space, then an approximated regression solution can be obtained with a faster speed. Moreover, the adaptive lasso is used to attain a sparse and globally optimal solution, which is important in variable selection. To complete the robust pattern classification task with corruptions, a correntropy-based and class-wise regression model is designed in this paper. It takes a smooth penalty instead of sparsity constraint in the regression coefficients, and it can be conducted in class-wise, thus it is more flexible in practice. Extensive experiments are conducted by using some real and benchmark data sets, e.g., high-dimensional facial images and gene microarray data, to evaluate the new algorithms. The new proposals attain competitive results and are compared with other state-of-the-art methods.