Lower bounds for finding stationary points I

• Published in: Mathematical programming Vol. 184; no. 1-2; pp. 71 - 120
• Main Authors: Carmon, Yair, Duchi, John C., Hinder, Oliver, Sidford, Aaron
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.11.2020; Springer Nature B.V
• Subjects: Algorithms; Calculus of Variations and Optimal Control; Optimization; Combinatorics; Complexity; Convex analysis; Derivatives; Electrical engineering; Full Length Paper; Lower bounds; Mathematical and Computational Physics; Mathematical Methods in Physics; Mathematical programming; Mathematics; Mathematics and Statistics; Mathematics of Computing; Numerical Analysis; Optimization; Queries; Regularization; Regularization methods; Theoretical; Dimension-free rates; Gradient descent; Cubic regularization of Newton’s method; 68Q25; Non-convex optimization; 90C30; Information-based complexity; 90C06; 90C26; 90C60

We prove lower bounds on the complexity of finding ϵ -stationary points (points x such that ‖ ∇ f ( x ) ‖ ≤ ϵ ) of smooth, high-dimensional, and potentially non-convex functions f . We consider oracle-based complexity measures, where an algorithm is given access to the value and all derivatives of f...