GAITA: A Gauss–Seidel iterative thresholding algorithm for ℓq regularized least squares regression

• Published in: Journal of computational and applied mathematics Vol. 319; pp. 220 - 235
• Main Authors: Zeng, Jinshan, Peng, Zhiming, Lin, Shaobo
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.08.2017
• Subjects: [formula omitted] regularized least squares; Gauss–Seidel; Iterative thresholding algorithm; Jacobi; Iterative thresholding algorithm; ℓq regularized least squares; Jacobi; Gauss–Seidel
• ISSN: 0377-0427; 1879-1778
• DOI: 10.1016/j.cam.2017.01.010

This paper studies the ℓq(0<q<1) regularized least squares regression (ℓqLS) problem, which arises in many applications of signal processing and machine learning. The iterative thresholding algorithm is an important algorithm for solving the ℓqLS problem, and can be viewed as a Jacobi-type iterative method. This paper proposes a Gauss–Seidel version of iterative thresholding algorithm called GAITA for solving the ℓqLS problem. Under certain conditions, we establish its global convergence11The global convergence in this paper is defined in the sense that the entire sequence converges regardless of the initial point., eventual linear rate, and the convergence to a local minimizer. Compared to the Jacobi counterpart, the proposed algorithm can allow larger step sizes and converge much faster. The effectiveness of the proposed algorithm is justified with numerical experiments on both synthetic data and real data.