A Lagrange–Newton algorithm for sparse nonlinear programming

• Published in: Mathematical programming Vol. 195; no. 1-2; pp. 903 - 928
• Main Authors: Zhao, Chen, Xiu, Naihua, Qi, Houduo, Luo, Ziyan
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.09.2022; Springer; Springer Nature B.V
• Subjects: Algorithms; Calculus of Variations and Optimal Control; Optimization; Combinatorics; Euler-Lagrange equation; Full Length Paper; Image processing; Iterative methods; Lagrangian function; Machine learning; Mathematical analysis; Mathematical and Computational Physics; Mathematical Methods in Physics; Mathematics; Mathematics and Statistics; Mathematics of Computing; Nonlinear equations; Nonlinear programming; Numerical Analysis; Signal processing; Theoretical; The Newton method; 90C30; 90C46; Locally quadratic convergence; Sparse nonlinear programming; 49M15; Lagrangian equation; Application
• ISSN: 0025-5610; 1436-4646
• DOI: 10.1007/s10107-021-01719-x

The sparse nonlinear programming (SNP) problem has wide applications in signal and image processing, machine learning and finance, etc. However, the computational challenge posed by SNP has not yet been well resolved due to the nonconvex and discontinuous ℓ 0 -norm involved. In this paper, we resolve this numerical challenge by developing a fast Newton-type algorithm. As a theoretical cornerstone, we establish a first-order optimality condition for SNP based on the concept of strong β -Lagrangian stationarity via the Lagrangian function, and reformulate it as a system of nonlinear equations called the Lagrangian equations. The nonsingularity of the corresponding Jacobian is discussed, based on which the Lagrange–Newton algorithm (LNA) is then proposed. Under mild conditions, we establish the locally quadratic convergence and its iterative complexity estimation. To further demonstrate the efficiency and superiority of our proposed algorithm, we apply LNA to two specific problems arising from compressed sensing and sparse high-order portfolio selection, in which significant benefits accrue from the restricted Newton step.