Branch-and-bound performance estimation programming: a unified methodology for constructing optimal optimization methods

• Published in: Mathematical programming Vol. 204; no. 1-2; pp. 567 - 639
• Main Authors: Das Gupta, Shuvomoy, Van Parys, Bart P. G., Ryu, Ernest K.
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.03.2024; Springer; Springer Nature B.V
• Subjects: Algorithms; Branch & bound algorithms; Calculus of Variations and Optimal Control; Optimization; Combinatorics; Customization; Full Length Paper; Mathematical and Computational Physics; Mathematical Methods in Physics; Mathematics; Mathematics and Statistics; Mathematics of Computing; Methodology; Methods; Numerical Analysis; Optimization; Theoretical; 68Q25; 90C30; 90C57; 90C25
• ISSN: 0025-5610; 1436-4646
• DOI: 10.1007/s10107-023-01973-1

We present the Branch-and-Bound Performance Estimation Programming (BnB-PEP), a unified methodology for constructing optimal first-order methods for convex and nonconvex optimization. BnB-PEP poses the problem of finding the optimal optimization method as a nonconvex but practically tractable quadratically constrained quadratic optimization problem and solves it to certifiable global optimality using a customized branch-and-bound algorithm. By directly confronting the nonconvexity, BnB-PEP offers significantly more flexibility and removes the many limitations of the prior methodologies. Our customized branch-and-bound algorithm, through exploiting specific problem structures, outperforms the latest off-the-shelf implementations by orders of magnitude, accelerating the solution time from hours to seconds and weeks to minutes. We apply BnB-PEP to several setups for which the prior methodologies do not apply and obtain methods with bounds that improve upon prior state-of-the-art results. Finally, we use the BnB-PEP methodology to find proofs with potential function structures, thereby systematically generating analytical convergence proofs.