A New Class of Alternating Proximal Minimization Algorithms with Costs-to-Move

• Published in: SIAM journal on optimization Vol. 18; no. 3; pp. 1061 - 1081
• Main Authors: Attouch, H., Redont, P., Soubeyran, A.
• Format: Journal Article
• Language: English
• Published: Philadelphia Society for Industrial and Applied Mathematics 01.01.2007
• Subjects: Algorithms; Convex analysis; Costs; Dynamical systems; Game theory; Hilbert space; Jargon
• ISSN: 1052-6234; 1095-7189
• DOI: 10.1137/060657248

Given two objective functions $f:\mathcal X\mapsto\R\cup\{+\infty\}$ and $g:\mathcal Y\mapsto\R\cup\{+\infty\}$ on abstract spaces $\mathcal X$ and $\mathcal Y$, and a coupling function $c:\mathcal X\times\mathcal Y\mapsto\R^+$, we introduce and study alternative minimization algorithms of the following type: $(x_0,y_0)\in\mathcal X\times\mathcal Y \mbox{ given; } (x_n,y_n)\rightarrow(x_{n+1},y_n)\rightarrow(x_{n+1},y_{n+1}) \mbox{ as follows: }\ \left\{\begin{array}{l} x_{n+1}\in\mbox{argmin} \{f(\xi)+\beta_n c(\xi,y_n)+\alpha_n h(x_n,\xi): \xi\in\mathcal X\},\ y_{n+1}\in\mbox{argmin} \{g(\eta)+\mu_n c(x_{n+1},\eta)+\nu_n k(y_n,\eta): \eta\in\mathcal Y\}. \end{array}\right.$ Their most original feature is the introduction of the terms $h:\mathcal X\times\mathcal X\mapsto\R^+$ and $k:\mathcal Y\times\mathcal Y\mapsto\R^+$ which are costs to change or to move (distance-like functions, relative entropies) accounting for various inertial, friction, or anchoring effects. These algorithms are studied in a general abstract framework. The introduction of the costs to change $h$ and $k$ leads to proximal minimizations with corresponding dissipative effects. As a result, the algorithms enjoy nice convergent properties. Coefficients $\alpha_n$, $\beta_n$, $\mu_n$, $\nu_n$ are nonnegative parameters. When taking $\alpha_n=\nu_n=0$ and quadratic costs on a Hilbert space, one recovers the classical alternating minimization algorithm, which itself is a natural extension of the alternating projection algorithm of von Neumann. A number of new significant results hold in general metric spaces. We pay particular attention to the following cases: (1.) $(\mathcal X,d_{\mathcal X})$ and $(\mathcal Y,d_{\mathcal Y})$ are complete metric spaces and $h\geq d_{\mathcal X}$, $k\geq d_{\mathcal Y}$ ("high local costs to move"); the algorithms then provide sequences that converge to Nash equilibria. (2.) $\mathcal X=\mathcal Y=\mathcal H$ is a Hilbert space, the costs to change are quadratic ("low local costs to move") and the functions $f,g:\mathcal H\mapsto\R\cup\{+\infty\}$ are closed, convex, proper; then some of the classical convergence theorems for alternating convex minimization algorithms, including those of Acker and Prestel, are properly extended with original proofs.