An Active-Set Proximal-Newton Algorithm for ℓ1 Regularized Optimization Problems with Box Constraints

• Published in: Journal of scientific computing Vol. 85; no. 3; p. 57
• Main Authors: Shen, Chungen, Xue, Wenjuan, Zhang, Lei-Hong, Wang, Baiyun
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.12.2020; Springer Nature B.V
• Subjects: Algorithms; Basic converters; Computational Mathematics and Numerical Analysis; Convergence; Convexity; Mathematical and Computational Engineering; Mathematical and Computational Physics; Mathematics; Mathematics and Statistics; Optimization; Quadratic programming; Theoretical; Variables; 90C30; Newton method; Active set; Box constraints; Proximity operator; regularization
• ISSN: 0885-7474; 1573-7691
• DOI: 10.1007/s10915-020-01364-0

In this paper, we propose an active-set proximal-Newton algorithm for solving ℓ 1 regularized convex/nonconvex optimization problems subject to box constraints. Our algorithm first relies on the KKT error to estimate the active and free variables, and then smoothly combines the proximal gradient iteration and the Newton iteration to efficiently pursue the convergence of the active and free variables, respectively. We show the global convergence without the convexity of the objective function. For some structured convex problems, we further design a safe screening procedure that is able to identify/remove active variables, and can be integrated into the basic active-set proximal-Newton algorithm to accelerate the convergence. The algorithm is evaluated on various synthetic and real data, and the efficiency is demonstrated particularly on ℓ 1 regularized convex/nonconvex quadratic programs and logistic regression problems.