Leveraging Non-uniformity in First-order Non-convex Optimization

• Published in: arXiv.org
• Main Authors: Mei, Jincheng, Gao, Yue, Dai, Bo, Szepesvari, Csaba, Schuurmans, Dale
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 02.06.2022
• Subjects: Algorithms; Computational geometry; Convergence; Convexity; Generalized linear models; Geometry; Lower bounds; Machine learning; Nonuniformity; Normalizing (statistics); Optimization; Smoothness; Statistical models
• ISSN: 2331-8422

Classical global convergence results for first-order methods rely on uniform smoothness and the \L{}ojasiewicz inequality. Motivated by properties of objective functions that arise in machine learning, we propose a non-uniform refinement of these notions, leading to \emph{Non-uniform Smoothness} (NS) and \emph{Non-uniform \L{}ojasiewicz inequality} (N\L{}). The new definitions inspire new geometry-aware first-order methods that are able to converge to global optimality faster than the classical \(\Omega(1/t^2)\) lower bounds. To illustrate the power of these geometry-aware methods and their corresponding non-uniform analysis, we consider two important problems in machine learning: policy gradient optimization in reinforcement learning (PG), and generalized linear model training in supervised learning (GLM). For PG, we find that normalizing the gradient ascent method can accelerate convergence to \(O(e^{-t})\) while incurring less overhead than existing algorithms. For GLM, we show that geometry-aware normalized gradient descent can also achieve a linear convergence rate, which significantly improves the best known results. We additionally show that the proposed geometry-aware descent methods escape landscape plateaus faster than standard gradient descent. Experimental results are used to illustrate and complement the theoretical findings.