Complexity of a projected Newton-CG method for optimization with bounds

• Published in: Mathematical programming Vol. 207; no. 1-2; pp. 107 - 144
• Main Authors: Xie, Yue, Wright, Stephen J.
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.09.2024; Springer Nature B.V
• Subjects: Algorithms; Calculus of Variations and Optimal Control; Optimization; Combinatorics; Complexity; Full Length Paper; Mathematical and Computational Physics; Mathematical Methods in Physics; Mathematics; Mathematics and Statistics; Mathematics of Computing; Newton methods; Numerical Analysis; Optimization; Theoretical; Conjugate gradient method; Complexity guarantees; 68Q25; Projected gradient method; 90C30; 90C06; Newton’s method; 49M15; 90C60; Nonconvex bound-constrained optimization
• ISSN: 0025-5610; 1436-4646
• DOI: 10.1007/s10107-023-02000-z

This paper describes a method for solving smooth nonconvex minimization problems subject to bound constraints with good worst-case complexity guarantees and practical performance. The method contains elements of two existing methods: the classical gradient projection approach for bound-constrained optimization and a recently proposed Newton-conjugate gradient algorithm for unconstrained nonconvex optimization. Using a new definition of approximate second-order optimality parametrized by some tolerance ϵ (which is compared with related definitions from previous works), we derive complexity bounds in terms of ϵ for both the number of iterations required and the total amount of computation. The latter is measured by the number of gradient evaluations or Hessian-vector products. We also describe illustrative computational results on several test problems from low-rank matrix optimization.