Gradient-Type Methods for Optimization Problems with Polyak-Łojasiewicz Condition: Early Stopping and Adaptivity to Inexactness Parameter

Due to its applications in many different places in machine learning and other connected engineering applications, the problem of minimization of a smooth function that satisfies the Polyak-Łojasiewicz condition receives much attention from researchers. Recently, for this problem, the authors of rec...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Kuruzov, Ilya A, Stonyakin, Fedor S, Alkousa, Mohammad S
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 08.12.2022
Subjects
Adaptive algorithms
Algorithms
Machine learning
Optimization
Online AccessGet full text

Cover

More Information
Summary:Due to its applications in many different places in machine learning and other connected engineering applications, the problem of minimization of a smooth function that satisfies the Polyak-Łojasiewicz condition receives much attention from researchers. Recently, for this problem, the authors of recent work proposed an adaptive gradient-type method using an inexact gradient. The adaptivity took place only with respect to the Lipschitz constant of the gradient. In this paper, for problems with the Polyak-Łojasiewicz condition, we propose a full adaptive algorithm, which means that the adaptivity takes place with respect to the Lipschitz constant of the gradient and the level of the noise in the gradient. We provide a detailed analysis of the convergence of the proposed algorithm and an estimation of the distance from the starting point to the output point of the algorithm. Numerical experiments and comparisons are presented to illustrate the advantages of the proposed algorithm in some examples.
ISSN:2331-8422