A framework of constraint preserving update schemes for optimization on Stiefel manifold

• Published in: Mathematical programming Vol. 153; no. 2; pp. 535 - 575
• Main Authors: Jiang, Bo, Dai, Yu-Hong
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.11.2015; Springer Nature B.V
• Subjects: Algorithms; Calculus of Variations and Optimal Control; Optimization; Combinatorics; Correlation; Energy conservation; Full Length Paper; Manifolds (mathematics); Mathematical and Computational Physics; Mathematical Methods in Physics; Mathematical models; Mathematical programming; Mathematics; Mathematics and Statistics; Mathematics of Computing; Methods; Minimization; Numerical Analysis; Optimization; Preserving; Principal components analysis; Searching; Studies; Theoretical; Heterogeneous quadratic functions; 65K05; Sphere constraint; Null space; Range space; Barzilai–Borwein-like method; Kohn–Sham total energy minimization; Adaptive nonmonotone line search; Orthogonality constraint; Feasible; 90C30; Stiefel manifold; 49Q99; Low-rank correlation matrix
• ISSN: 0025-5610; 1436-4646
• DOI: 10.1007/s10107-014-0816-7

This paper considers optimization problems on the Stiefel manifold X T X = I p , where X ∈ R n × p is the variable and I p is the p -by- p identity matrix. A framework of constraint preserving update schemes is proposed by decomposing each feasible point into the range space of X and the null space of X T . While this general framework can unify many existing schemes, a new update scheme with low complexity cost is also discovered. Then we study a feasible Barzilai–Borwein-like method under the new update scheme. The global convergence of the method is established with an adaptive nonmonotone line search. The numerical tests on the nearest low-rank correlation matrix problem, the Kohn–Sham total energy minimization and a specific problem from statistics demonstrate the efficiency of the new method. In particular, the new method performs remarkably well for the nearest low-rank correlation matrix problem in terms of speed and solution quality and is considerably competitive with the widely used SCF iteration for the Kohn–Sham total energy minimization.