Generalized principal component analysis (GPCA)

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 27; no. 12; pp. 1945 - 1959
• Main Authors: Vidal, R., Yi Ma, Sastry, S.
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.12.2005; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Application software; Applied sciences; Artificial Intelligence; Cluster Analysis; Clustering algorithms; Computer science; control theory; systems; Computer Simulation; Computer vision; Data points; Derivatives; dimensionality reduction; dynamic scenes and motion segmentation; Exact sciences and technology; Image Enhancement - methods; Image Interpretation, Computer-Assisted - methods; Imaging, Three-Dimensional - methods; Index Terms- Principal component analysis (PCA); Information Storage and Retrieval - methods; Iterative algorithms; Kernel; Machine learning; Mathematical analysis; Mathematical models; Matrix decomposition; Models, Statistical; Motion segmentation; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; Polynomials; Principal Component Analysis; Principal components analysis; Segmentation; Studies; subspace segmentation; Subspaces; temporal video segmentation; Three dimensional; Vectors (mathematics); Veronese map; subspace segmentation; Computer vision; dynamic scenes and motion segmentation; Veronese map; temporal video segmentation; Multiple view; Index Terms-Principal component analysis (PCA); Image segmentation; Dimension reduction; dimensionality reduction; Multidimensional database; Scene analysis; Tridimensional image; EM algorithm; Pattern analysis; Principal component analysis
• ISSN: 0162-8828; 1939-3539
• DOI: 10.1109/TPAMI.2005.244

This paper presents an algebro-geometric solution to the problem of segmenting an unknown number of subspaces of unknown and varying dimensions from sample data points. We represent the subspaces with a set of homogeneous polynomials whose degree is the number of subspaces and whose derivatives at a data point give normal vectors to the subspace passing through the point. When the number of subspaces is known, we show that these polynomials can be estimated linearly from data; hence, subspace segmentation is reduced to classifying one point per subspace. We select these points optimally from the data set by minimizing certain distance function, thus dealing automatically with moderate noise in the data. A basis for the complement of each subspace is then recovered by applying standard PCA to the collection of derivatives (normal vectors). Extensions of GPCA that deal with data in a high-dimensional space and with an unknown number of subspaces are also presented. Our experiments on low-dimensional data show that GPCA outperforms existing algebraic algorithms based on polynomial factorization and provides a good initialization to iterative techniques such as k-subspaces and expectation maximization. We also present applications of GPCA to computer vision problems such as face clustering, temporal video segmentation, and 3D motion segmentation from point correspondences in multiple affine views.