A unified convergence rate analysis of the accelerated smoothed gap reduction algorithm

• Published in: Optimization letters Vol. 16; no. 4; pp. 1235 - 1257
• Main Author: Tran-Dinh, Quoc
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.05.2022
• Subjects: Computational Intelligence; Mathematics; Mathematics and Statistics; Numerical and Computational Physics; Operations Research/Decision Theory; Optimization; Original Paper; Simulation; Nesterov’s smoothing technique; Accelerated smoothed gap reduction; Primal–dual algorithm; Convex–concave saddle-point problem
• ISSN: 1862-4472; 1862-4480
• DOI: 10.1007/s11590-021-01775-4

In this paper, we develop a unified convergence analysis framework for the Accelerated Smoothed GAp ReDuction algorithm (ASGARD) introduced in Tran-Dinh et al. (SIAM J Optim 28(1):96–134, 2018). Unlike Tran-Dinh et al. (SIAM J Optim 28(1):96–134, 2018), the new analysis covers three settings in a single algorithm: general convexity, strong convexity, and strong convexity and smoothness. Moreover, we establish the convergence guarantees on three criteria: (i) gap function, (ii) primal objective residual, and (iii) dual objective residual. Our convergence rates are optimal (up to a constant factor) in all cases. While the convergence rate on the primal objective residual for the general convex case has been established in Tran-Dinh et al. (SIAM J Optim 28(1):96–134, 2018), we prove additional convergence rates on the gap function and the dual objective residual. The analysis for the last two cases is completely new. Our results provide a complete picture on the convergence guarantees of ASGARD. Finally, we present four different numerical experiments on a representative optimization model to verify our algorithm and compare it with the well-known Nesterov’s smoothing algorithm.