A self-consistent-field iteration for MAXBET with an application to multi-view feature extraction

• Published in: Advances in computational mathematics Vol. 48; no. 2
• Main Authors: Ma, Xijun, Shen, Chungen, Wang, Li, Zhang, Lei-Hong, Li, Ren-Cang
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.04.2022; Springer Nature B.V
• Subjects: Computational mathematics; Computational Mathematics and Numerical Analysis; Computational Science and Engineering; Correlation analysis; Eigenvalues; Eigenvectors; Feature extraction; Iterative methods; Manifolds (mathematics); Mathematical and Computational Biology; Mathematical Modeling and Industrial Mathematics; Mathematics; Mathematics and Statistics; Orthogonality; Principal components analysis; Quadratic equations; Random variables; Visualization; 90C20; Self-consistent-field iteration; Multi-view feature extraction; 65F10; Stiefel manifold; 90C06; 65F15; Multi-view canonical correlation analysis; 65F35; Nonlinear eigenvalue problem; MAXBET
• ISSN: 1019-7168; 1572-9044
• DOI: 10.1007/s10444-022-09929-3

As an extension of the traditional principal component analysis, the multi-view canonical correlation analysis (MCCA) aims at reducing m high dimensional random variables s i ∈ ℝ n i ( i = 1 , 2 , … , m ) by proper projection matrices X i ∈ ℝ n i × ℓ so that the m reduced ones y i = X i T s i ∈ ℝ ℓ have the “maximal correlation.” Various measures of the correlation for y i ( i = 1,2,…, m ) in MCCA have been proposed. One of the earliest criteria is the sum of all traces of pair-wise correlation matrices between y i and y j subject to the orthogonality constraints on X i , i = 1,2,…, m . The resulting problem is to maximize a homogeneous quadratic function over the product of Stiefel manifolds and is referred to as the MAXBET problem . In this paper, the problem is first reformulated as a coupled nonlinear eigenvalue problem with eigenvector dependency (NEPv) and then solved by a novel self-consistent-field (SCF) iteration. Global and local convergences of the SCF iteration are studied and proven computational techniques in the standard eigenvalue problem are incorporated to yield more practical implementations. Besides the preliminary numerical evaluations on various types of synthetic problems, the efficiency of the SCF iteration is also demonstrated in an application to multi-view feature extraction for unsupervised learning.