Riemannian proximal gradient methods

• Published in: Mathematical programming Vol. 194; no. 1-2; pp. 371 - 413
• Main Authors: Huang, Wen, Wei, Ke
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.07.2022; Springer; Springer Nature B.V
• Subjects: Algorithms; Analysis; Calculus of Variations and Optimal Control; Optimization; Combinatorics; Convergence; Convexity; Full Length Paper; Manifolds (mathematics); Mathematical and Computational Physics; Mathematical Methods in Physics; Mathematics; Mathematics and Statistics; Mathematics of Computing; Methods; Numerical Analysis; Optimization; Theoretical; Riemannian proximal gradient; Stiefel manifold; 90–08; 62–08; Sparse PCA; 90C26; Oblique manifold; Riemannian optimization
• ISSN: 0025-5610; 1436-4646
• DOI: 10.1007/s10107-021-01632-3

In the Euclidean setting the proximal gradient method and its accelerated variants are a class of efficient algorithms for optimization problems with decomposable objective. In this paper, we develop a Riemannian proximal gradient method (RPG) and its accelerated variant (ARPG) for similar problems but constrained on a manifold. The global convergence of RPG is established under mild assumptions, and the O (1/ k ) is also derived for RPG based on the notion of retraction convexity. If assuming the objective function obeys the Rimannian Kurdyka–Łojasiewicz (KL) property, it is further shown that the sequence generated by RPG converges to a single stationary point. As in the Euclidean setting, local convergence rate can be established if the objective function satisfies the Riemannian KL property with an exponent. Moreover, we show that the restriction of a semialgebraic function onto the Stiefel manifold satisfies the Riemannian KL property, which covers for example the well-known sparse PCA problem. Numerical experiments on random and synthetic data are conducted to test the performance of the proposed RPG and ARPG.