Subexponential-Time Algorithms for Sparse PCA

• Published in: Foundations of computational mathematics Vol. 24; no. 3; pp. 865 - 914
• Main Authors: Ding, Yunzi, Kunisky, Dmitriy, Wein, Alexander S., Bandeira, Afonso S.
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.06.2024; Springer Nature B.V
• Subjects: Algorithms; Applications of Mathematics; Computational efficiency; Computer Science; Economics; Likelihood ratio; Linear and Multilinear Algebras; Math Applications in Computer Science; Mathematics; Mathematics and Statistics; Matrix Theory; Numerical Analysis; Polynomials; Principal components analysis; Recovery; Search algorithms; Signal to noise ratio; Tradeoffs; Low-degree likelihood ratio; Sparse PCA; Computational complexity; 62F03; 68Q87; 68Q17
• ISSN: 1615-3375; 1615-3383
• DOI: 10.1007/s10208-023-09603-0

We study the computational cost of recovering a unit-norm sparse principal component x ∈ R n planted in a random matrix, in either the Wigner or Wishart spiked model (observing either W + λ x x ⊤ with W drawn from the Gaussian orthogonal ensemble, or N independent samples from N ( 0 , I n + β x x ⊤ ) , respectively). Prior work has shown that when the signal-to-noise ratio ( λ or β N / n , respectively) is a small constant and the fraction of nonzero entries in the planted vector is ‖ x ‖ 0 / n = ρ , it is possible to recover x in polynomial time if ρ ≲ 1 / n . While it is possible to recover x in exponential time under the weaker condition ρ ≪ 1 , it is believed that polynomial-time recovery is impossible unless ρ ≲ 1 / n . We investigate the precise amount of time required for recovery in the “possible but hard” regime 1 / n ≪ ρ ≪ 1 by exploring the power of subexponential-time algorithms, i.e., algorithms running in time exp ( n δ ) for some constant δ ∈ ( 0 , 1 ) . For any 1 / n ≪ ρ ≪ 1 , we give a recovery algorithm with runtime roughly exp ( ρ 2 n ) , demonstrating a smooth tradeoff between sparsity and runtime. Our family of algorithms interpolates smoothly between two existing algorithms: the polynomial-time diagonal thresholding algorithm and the exp ( ρ n ) -time exhaustive search algorithm. Furthermore, by analyzing the low-degree likelihood ratio, we give rigorous evidence suggesting that the tradeoff achieved by our algorithms is optimal.