The 2-Coordinate Descent Method for Solving Double-Sided Simplex Constrained Minimization Problems

• Published in: Journal of optimization theory and applications Vol. 162; no. 3; pp. 892 - 919
• Main Author: Beck, Amir
• Format: Journal Article
• Language: English
• Published: Boston Springer US 01.09.2014; Springer Nature B.V
• Subjects: Algorithms; Applications of Mathematics; Blocking; Calculus of Variations and Optimal Control; Optimization; Convergence; Decomposition; Descent; Engineering; Iterative methods; Mathematical analysis; Mathematical models; Mathematics; Mathematics and Statistics; Methods; Minimization; Operations research; Operations Research/Decision Theory; Optimization; Optimization algorithms; Problem solving; Studies; Support vector machines; Theory of Computation; Variables; Block descent method; Nonconvex optimization; Rate of convergence; Simplex-type constraints
• ISSN: 0022-3239; 1573-2878
• DOI: 10.1007/s10957-013-0491-5

This paper considers the problem of minimizing a continuously differentiable function with a Lipschitz continuous gradient subject to a single linear equality constraint and additional bound constraints on the decision variables. We introduce and analyze several variants of a 2-coordinate descent method: a block descent method that performs an optimization step with respect to only two variables at each iteration. Based on two new optimality measures, we establish convergence to stationarity points for general nonconvex objective functions. In the convex case, when all the variables are lower bounded but not upper bounded, we show that the sequence of function values converges at a sublinear rate. Several illustrative numerical examples demonstrate the effectiveness of the method.