Asynchronous parallel algorithms for nonconvex optimization

• Published in: Mathematical programming Vol. 184; no. 1-2; pp. 121 - 154
• Main Authors: Cannelli, Loris, Facchinei, Francisco, Kungurtsev, Vyacheslav, Scutari, Gesualdo
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.11.2020; Springer Nature B.V
• Subjects: Algorithms; Approximation; Calculus of Variations and Optimal Control; Optimization; Combinatorics; Communication; Full Length Paper; Mathematical and Computational Physics; Mathematical Methods in Physics; Mathematical programming; Mathematics; Mathematics and Statistics; Mathematics of Computing; Numerical Analysis; Optimization; Probabilistic models; Theoretical; Variables; Workers; Nonconvex constraints; 90C30; 65K05; Asynchronous algorithms; 68W10; Parallel methods; Probabilistic model; 90C26
• ISSN: 0025-5610; 1436-4646
• DOI: 10.1007/s10107-019-01408-w

We propose a new asynchronous parallel block-descent algorithmic framework for the minimization of the sum of a smooth nonconvex function and a nonsmooth convex one, subject to both convex and nonconvex constraints. The proposed framework hinges on successive convex approximation techniques and a novel probabilistic model that captures key elements of modern computational architectures and asynchronous implementations in a more faithful way than current state-of-the-art models. Other key features of the framework are: (1) it covers in a unified way several specific solution methods; (2) it accommodates a variety of possible parallel computing architectures; and (3) it can deal with nonconvex constraints. Almost sure convergence to stationary solutions is proved, and theoretical complexity results are provided, showing nearly ideal linear speedup when the number of workers is not too large.