CP decomposition for tensors via alternating least squares with QR decomposition

• Published in: Numerical linear algebra with applications Vol. 30; no. 6
• Main Authors: Minster, Rachel, Viviano, Irina, Liu, Xiaotian, Ballard, Grey
• Format: Journal Article
• Language: English
• Published: Oxford Wiley Subscription Services, Inc 01.12.2023; Wiley
• Subjects: Algorithms; CANDECOMP/PARAFAC; canonical polyadic tensor decomposition; Decomposition; Least squares; Machine learning; Mathematical analysis; MATHEMATICS AND COMPUTING; Multidimensional data; multilinear algebra; numerical stability; Singular value decomposition; Tensors
• ISSN: 1070-5325; 1099-1506
• DOI: 10.1002/nla.2511

Abstract The CP tensor decomposition is used in applications such as machine learning and signal processing to discover latent low‐rank structure in multidimensional data. Computing a CP decomposition via an alternating least squares (ALS) method reduces the problem to several linear least squares problems. The standard way to solve these linear least squares subproblems is to use the normal equations, which inherit special tensor structure that can be exploited for computational efficiency. However, the normal equations are sensitive to numerical ill‐conditioning, which can compromise the results of the decomposition. In this paper, we develop versions of the CP‐ALS algorithm using the QR decomposition and the singular value decomposition, which are more numerically stable than the normal equations, to solve the linear least squares problems. Our algorithms utilize the tensor structure of the CP‐ALS subproblems efficiently, have the same complexity as the standard CP‐ALS algorithm when the input is dense and the rank is small, and are shown via examples to produce more stable results when ill‐conditioning is present. Our MATLAB implementation achieves the same running time as the standard algorithm for small ranks, and we show that the new methods can obtain lower approximation error.