Geometrical interpretation of the predictor-corrector type algorithms in structured optimization problems

• Published in: Optimization Vol. 55; no. 5-6; pp. 481 - 503
• Main Authors: Daniilidis, Aris, Hare, Warren, Malick, Jérôme
• Format: Journal Article
• Language: English
• Published: Philadelphia Taylor & Francis Group 01.10.2006; Taylor & Francis LLC; Taylor & Francis
• Subjects: Algorithms; Mathematics; Mathematics Subject Classification 2000: Primary: 49J52; Newton-type methods; Optimization; Optimization algorithms; Optimization and Control; partly smooth function; Proximal algorithm; Riemannian gradient; Secondary: 90C26; Studies; Topological manifolds; U-Lagrangian
• ISSN: 0233-1934; 1029-4945
• DOI: 10.1080/02331930600815884

It has been observed that in many optimization problems, nonsmooth objective functions often appear smooth on naturally arising manifolds. This has led to the development of optimization algorithms which attempt to exploit this smoothness. Many of these algorithms follow the same two-step pattern: first to predict a direction of decrease, and second to make a correction step to return to the manifold. In this article, we examine some of the theoretical components used in such predictor-corrector methods. We begin our examination under the minimal assumption that the restriction of the function to the manifold is smooth. At the second stage, we add the condition of 'partial smoothness' relative to the manifold. Finally, we examine the case when the function is both 'prox-regular' and partly smooth. In this final setting, we show that the proximal point mapping can be used to return to the manifold, and argue that returning in this manner is preferable to returning via the projection mapping. We finish by developing sufficient conditions for quadratic convergence of predictor-corrector methods using a proximal point correction step. ¶Dedicated to Professor D. Pallaschke for the occasion of his 65th birthday.