Orthogonal Low Rank Tensor Approximation: Alternating Least Squares Method and Its Global Convergence

• Published in: SIAM journal on matrix analysis and applications Vol. 36; no. 1; pp. 1 - 19
• Main Authors: Wang, Liqi, Chu, Moody T., Yu, Bo
• Format: Journal Article
• Language: English
• Published: 01.01.2015
• Subjects: Algebra; Approximation; Convergence; Decomposition; Least squares method; Mathematical analysis; Orthogonality; Tensors
• ISSN: 0895-4798; 1095-7162
• DOI: 10.1137/130943133

With the notable exceptions of two cases---that tensors of order 2, namely, matrices, always have best approximations of arbitrary low ranks and that tensors of any order always have the best rank-1 approximation, it is known that high-order tensors may fail to have best low rank approximations. When the condition of orthogonality is imposed, even under the modest assumption of semiorthogonality where only one set of components in the decomposed rank-1 tensors is required to be mutually perpendicular, the situation is changed completely---orthogonal low rank approximations always exist. The purpose of this paper is to discuss the best low rank approximation subject to semiorthogonality. The conventional high-order power method is modified to address the desirable orthogonality via the polar decomposition. Algebraic geometry technique is employed to show that for almost all tensors the orthogonal alternating least squares method converges globally.