Quadratic optimization for simultaneous matrix diagonalization

• Published in: IEEE transactions on signal processing Vol. 54; no. 9; pp. 3270 - 3278
• Main Authors: Vollgraf, R., Obermayer, K.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.09.2006; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Applied sciences; Blind source separation; Blind source separation (BSS); Constrictions; Coordinate measuring machines; Detection, estimation, filtering, equalization, prediction; Eigenvalues and eigenfunctions; Exact sciences and technology; Information, signal and communications theory; Jacobian matrices; joint diagonalization; Mathematical analysis; Matrices; Matrix methods; Miscellaneous; Optimization; Optimization algorithms; Principal component analysis; quadratic optimization; Signal and communications theory; Signal processing; Signal processing algorithms; Signal, noise; Source separation; Statistics; Studies; Symmetric matrices; Telecommunications and information theory; Tensile stress; Transformations; Matrix diagonalization; Second order; Hermite interpolation; Blind source separation; Blind source separation (BSS); quadratic optimization; Implementation; Optimization; Orthogonal transformation; Least squares method; Signal processing; joint diagonalization; Objective function; Fast algorithm
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2006.877673

Simultaneous diagonalization of a set of matrices is a technique that has numerous applications in statistical signal processing and multivariate statistics. Although objective functions in a least-squares sense can be easily formulated, their minimization is not trivial, because constraints and fourth-order terms are usually involved. Most known optimization algorithms are, therefore, subject to certain restrictions on the class of problems: orthogonal transformations, sets of symmetric, Hermitian or positive definite matrices, to name a few. In this paper, we present a new algorithm called QDIAG that splits the overall optimization problem into a sequence of simpler second order subproblems. There are no restrictions imposed on the transformation matrix, which may be nonorthogonal, indefinite, or even rectangular, and there are no restrictions regarding the symmetry and definiteness of the matrices to be diagonalized, except for one of them. We apply the new method to second-order blind source separation and show that the algorithm converges fast and reliably. It allows for an implementation with a complexity independent of the number of matrices and, therefore, is particularly suitable for problems dealing with large sets of matrices