Eigenvectors from Eigenvalues Sparse Principal Component Analysis

• Published in: Journal of computational and graphical statistics Vol. 31; no. 2; pp. 486 - 501
• Main Author: Robert Frost, H.
• Format: Journal Article
• Language: English
• Published: United States Taylor & Francis 03.04.2022; Taylor & Francis Ltd
• Subjects: Dimensional analysis; Eigenvalues; Eigenvector-eigenvalue identity; Eigenvectors; Errors; Parameter identification; Principal component analysis; Principal components analysis; Resampling; Sparse eigenvalue decomposition; Sparse principal component analysis; eigenvector-eigenvalue identity; sparse principal component analysis; principal component analysis; sparse eigenvalue decomposition
• ISSN: 1061-8600; 1537-2715
• DOI: 10.1080/10618600.2021.1987254

We present a novel technique for sparse principal component analysis. This method, named eigenvectors from eigenvalues sparse principal component analysis (EESPCA), is based on the formula for computing squared eigenvector loadings of a Hermitian matrix from the eigenvalues of the full matrix and associated sub-matrices. We explore two versions of the EESPCA method: a version that uses a fixed threshold for inducing sparsity and a version that selects the threshold via cross-validation. Relative to the state-of-the-art sparse PCA methods of Witten et al., Yuan and Zhang, and Tan et al., the fixed threshold EESPCA technique offers an order-of-magnitude improvement in computational speed, does not require estimation of tuning parameters via cross-validation, and can more accurately identify true zero principal component loadings across a range of data matrix sizes and covariance structures. Importantly, the EESPCA method achieves these benefits while maintaining out-of-sample reconstruction error and PC estimation error close to the lowest error generated by all evaluated approaches. EESPCA is a practical and effective technique for sparse PCA with particular relevance to computationally demanding statistical problems such as the analysis of high-dimensional datasets or application of statistical techniques like resampling that involve the repeated calculation of sparse PCs. Supplementary materials for this article are available online.