Block-coordinate and incremental aggregated proximal gradient methods for nonsmooth nonconvex problems

This paper analyzes block-coordinate proximal gradient methods for minimizing the sum of a separable smooth function and a (nonseparable) nonsmooth function, both of which are allowed to be nonconvex. The main tool in our analysis is the forward-backward envelope, which serves as a particularly suit...

Full description

Saved in:
Bibliographic Details
Published inMathematical programming Vol. 193; no. 1; pp. 195 - 224
Main Authors Latafat, Puya, Themelis, Andreas, Patrinos, Panagiotis
Format Journal Article
LanguageEnglish
Published Berlin/Heidelberg Springer Berlin Heidelberg 01.05.2022
Springer
Springer Nature B.V
Subjects
Algorithms
Analysis
Calculus of Variations and Optimal Control; Optimization
Combinatorics
Continuity (mathematics)
Convergence
Convexity
Cost function
Full Length Paper
Liapunov functions
Mathematical and Computational Physics
Mathematical Methods in Physics
Mathematics
Mathematics and Statistics
Mathematics of Computing
Methods
Numerical Analysis
Theoretical
Forward-backward envelope
49J53
90C25
49J52
Block-coordinate updates
90C06
KL inequality
90C26
Nonsmooth nonconvex optimization
Online AccessGet full text

Cover

More Information
Summary:This paper analyzes block-coordinate proximal gradient methods for minimizing the sum of a separable smooth function and a (nonseparable) nonsmooth function, both of which are allowed to be nonconvex. The main tool in our analysis is the forward-backward envelope, which serves as a particularly suitable continuous and real-valued Lyapunov function. Global and linear convergence results are established when the cost function satisfies the Kurdyka–Łojasiewicz property without imposing convexity requirements on the smooth function. Two prominent special cases of the investigated setting are regularized finite sum minimization and the sharing problem; in particular, an immediate byproduct of our analysis leads to novel convergence results and rates for the popular Finito/MISO algorithm in the nonsmooth and nonconvex setting with very general sampling strategies.
ISSN:0025-5610
1436-4646
DOI:10.1007/s10107-020-01599-7