Trace Ratio Problem Revisited

• Published in: IEEE transactions on neural networks Vol. 20; no. 4; pp. 729 - 735
• Main Authors: Jia, Yangqing, Nie, Feiping, Zhang, Changshui
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.04.2009; Institute of Electrical and Electronics Engineers
• Subjects: Algorithm design and analysis; Algorithms; Applied sciences; Approximation; Artificial intelligence; Automation; Computer science; control theory; systems; Convergence; Dimensionality reduction; eigenvalue perturbation; Eigenvalues; Eigenvalues and eigenfunctions; Equivalence; Exact sciences and technology; Information science; Intelligent systems; Linear discriminant analysis; Machine learning; Machine learning algorithms; Neural networks; Newton-Raphson method; Optimization; Pattern recognition; Principal component analysis; Reduction; trace ratio (TR); Perturbation theory; Dimensionality reduction; Newton-Raphson method; Global optimum; Pattern recognition; Neural network; Modeling; Global solution; Newton Raphson method; Dimension reduction; Perturbation method; eigenvalue perturbation; Eigenvalue problem; trace ratio (TR); Artificial intelligence; Mathematical programming
• ISSN: 1045-9227; 1941-0093
• DOI: 10.1109/TNN.2009.2015760

Dimensionality reduction is an important issue in many machine learning and pattern recognition applications, and the trace ratio (TR) problem is an optimization problem involved in many dimensionality reduction algorithms. Conventionally, the solution is approximated via generalized eigenvalue decomposition due to the difficulty of the original problem. However, prior works have indicated that it is more reasonable to solve it directly than via the conventional way. In this brief, we propose a theoretical overview of the global optimum solution to the TR problem via the equivalent trace difference problem. Eigenvalue perturbation theory is introduced to derive an efficient algorithm based on the Newton-Raphson method. Theoretical issues on the convergence and efficiency of our algorithm compared with prior literature are proposed, and are further supported by extensive empirical results.