Lower bounds for non-convex stochastic optimization

• Published in: Mathematical programming Vol. 199; no. 1-2; pp. 165 - 214
• Main Authors: Arjevani, Yossi, Carmon, Yair, Duchi, John C., Foster, Dylan J., Srebro, Nathan, Woodworth, Blake
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.05.2023; Springer; Springer Nature B.V
• Subjects: Algorithms; Analysis; Calculus of Variations and Optimal Control; Optimization; Combinatorics; Full Length Paper; Lower bounds; Mathematical and Computational Physics; Mathematical Methods in Physics; Mathematics; Mathematics and Statistics; Mathematics of Computing; Minimax technique; Numerical Analysis; Optimization; Queries; Smoothness; Theoretical; 90C06; 68Q25; 90C15; 90C26; 90C30; 90C60

We lower bound the complexity of finding ϵ -stationary points (with gradient norm at most ϵ ) using stochastic first-order methods. In a well-studied model where algorithms access smooth, potentially non-convex functions through queries to an unbiased stochastic gradient oracle with bounded variance...