Flexible Manifold Embedding: A Framework for Semi-Supervised and Unsupervised Dimension Reduction

• Published in: IEEE transactions on image processing Vol. 19; no. 7; pp. 1921 - 1932
• Main Authors: Feiping Nie, Dong Xu, Tsang, Ivor Wai-Hung, Changshui Zhang
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.07.2010; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Automation; Data points; Dimension reduction; Economic models; Exact sciences and technology; Face recognition; Image processing; Information science; Information, signal and communications theory; Intelligent systems; Labels; Laboratories; Learning; Linear discriminant analysis; Linear regression; manifold embedding; Manifolds; Mathematical models; Pattern recognition; Principal component analysis; Reduction; Regression; Regression analysis; Residues; Scattering; semi-supervised learning; Semisupervised learning; Signal and communications theory; Signal processing; Signal representation. Spectral analysis; Signal, noise; Studies; Telecommunications and information theory; Biometrics; Face recognition; Image processing; Linear regression; Regression analysis; Unsupervised classification; Pattern recognition; Algorithm; Signal classification; Mismatching; Modeling; Dimension reduction; manifold embedding; Supervised learning; Database; Objective function; semi-supervised learning; Automatic recognition
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2010.2044958

We propose a unified manifold learning framework for semi-supervised and unsupervised dimension reduction by employing a simple but effective linear regression function to map the new data points. For semi-supervised dimension reduction, we aim to find the optimal prediction labels F for all the training samples X, the linear regression function h(X) and the regression residue F 0 = F - h(X) simultaneously. Our new objective function integrates two terms related to label fitness and manifold smoothness as well as a flexible penalty term defined on the residue F 0 . Our Semi-Supervised learning framework, referred to as flexible manifold embedding (FME), can effectively utilize label information from labeled data as well as a manifold structure from both labeled and unlabeled data. By modeling the mismatch between h(X) and F, we show that FME relaxes the hard linear constraint F = h(X) in manifold regularization (MR), making it better cope with the data sampled from a nonlinear manifold. In addition, we propose a simplified version (referred to as FME/U) for unsupervised dimension reduction. We also show that our proposed framework provides a unified view to explain and understand many semi-supervised, supervised and unsupervised dimension reduction techniques. Comprehensive experiments on several benchmark databases demonstrate the significant improvement over existing dimension reduction algorithms.