On the superiority of PGMs to PDCAs in nonsmooth nonconvex sparse regression

• Published in: Optimization letters Vol. 15; no. 8; pp. 2831 - 2860
• Main Authors: Nakayama, Shummin, Gotoh, Jun-ya
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.11.2021
• Subjects: Computational Intelligence; Mathematics; Mathematics and Statistics; Numerical and Computational Physics; Operations Research/Decision Theory; Optimization; Original Paper; Simulation; Proximal alternating linearized minimization; DC algorithms; Critical points; Proximal gradient method; D-stationary points
• ISSN: 1862-4472; 1862-4480
• DOI: 10.1007/s11590-021-01716-1

This paper conducts a comparative study of proximal gradient methods (PGMs) and proximal DC algorithms (PDCAs) for sparse regression problems which can be cast as Difference-of-two-Convex-functions (DC) optimization problems. It has been shown that for DC optimization problems, both General Iterative Shrinkage and Thresholding algorithm (GIST), a modified version of PGM, and PDCA converge to critical points. Recently some enhanced versions of PDCAs are shown to converge to d-stationary points, which are stronger necessary condition for local optimality than critical points. In this paper we claim that without any modification, PGMs converge to a d-stationary point not only to DC problems but also to more general nonsmooth nonconvex problems under some technical assumptions. While the convergence to d-stationary points is known for the case where the step size is small enough, the finding of this paper is valid also for extended versions such as GIST and its alternating optimization version, which is to be developed in this paper. Numerical results show that among several algorithms in the two categories, modified versions of PGM perform best among those not only in solution quality but also in computation time.