Max-Min Distance Analysis by Using Sequential SDP Relaxation for Dimension Reduction

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 33; no. 5; pp. 1037 - 1050
• Main Authors: Bian, Wei, Tao, Dacheng
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.05.2011; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithm design and analysis; Algorithms; Applied sciences; Approximation algorithms; Artificial intelligence; Classification; Computer science; control theory; systems; convex relaxation; Covariance matrix; Criteria; Data processing. List processing. Character string processing; Data visualization; dimension reduction; Discriminant analysis; Eigenvalues and eigenfunctions; Exact sciences and technology; Fisher's linear discriminant analysis; Gaussian; Kernel; Memory organisation. Data processing; Optimization; Optimized production technology; Pattern analysis; pattern classification; Reduction; Software; Studies; Discriminant analysis; Optimization; Fisher information; Sequential analysis; Dimension reduction; Gaussian process; Vector space; Pattern classification; Data visualization; Minimax method; Data distribution; Minimal distance; convex relaxation; Fisher's linear discriminant analysis; Mathematical programming
• ISSN: 0162-8828; 1939-3539
• DOI: 10.1109/TPAMI.2010.189

We propose a new criterion for discriminative dimension reduction, max-min distance analysis (MMDA). Given a data set with C classes, represented by homoscedastic Gaussians, MMDA maximizes the minimum pairwise distance of these C classes in the selected low-dimensional subspace. Thus, unlike Fisher's linear discriminant analysis (FLDA) and other popular discriminative dimension reduction criteria, MMDA duly considers the separation of all class pairs. To deal with general case of data distribution, we also extend MMDA to kernel MMDA (KMMDA). Dimension reduction via MMDA/KMMDA leads to a nonsmooth max-min optimization problem with orthonormal constraints. We develop a sequential convex relaxation algorithm to solve it approximately. To evaluate the effectiveness of the proposed criterion and the corresponding algorithm, we conduct classification and data visualization experiments on both synthetic data and real data sets. Experimental results demonstrate the effectiveness of MMDA/KMMDA associated with the proposed optimization algorithm.