Manifold-Manifold Distance and its Application to Face Recognition With Image Sets

• Published in: IEEE transactions on image processing Vol. 21; no. 10; pp. 4466 - 4479
• Main Authors: Wang, Ruiping, Shan, Shiguang, Chen, Xilin, Dai, Qionghai, Gao, Wen
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.10.2012; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Applied sciences; Biometric Identification - methods; Cluster Analysis; Clustering algorithms; Computational modeling; Data models; Databases, Factual; Exact sciences and technology; Face - anatomy & histology; Face recognition; Face recognition with image sets; hierarchical divisive clustering; Humans; Image processing; Image Processing, Computer-Assisted - methods; Information, signal and communications theory; manifold-manifold distance (MMD); Manifolds; Mathematical models; Multilevel; Pattern recognition; principal angles; Probes; set similarity measure; Signal and communications theory; Signal processing; Signal representation. Spectral analysis; Signal, noise; Similarity; Subspaces; Tasks; Telecommunications and information theory; Vectors; Biometrics; Performance evaluation; Discriminant analysis; Automatic classification; State of the art; manifold―manifold distance (MMD); Multipoint method; Face recognition; Similarity; Image processing; Hierarchical classification; Subspace method; Pattern recognition; Signal classification; Modeling; Linear model; set similarity measure; principal angles; hierarchical divisive clustering; Automatic recognition; Multilevel system; Face recognition with image sets
• DOI: 10.1109/TIP.2012.2206039
• ISSN: 1057-7149; 1941-0042

In this paper, we address the problem of classifying image sets for face recognition, where each set contains images belonging to the same subject and typically covering large variations. By modeling each image set as a manifold, we formulate the problem as the computation of the distance between two manifolds, called manifold-manifold distance (MMD). Since an image set can come in three pattern levels, point, subspace, and manifold, we systematically study the distance among the three levels and formulate them in a general multilevel MMD framework. Specifically, we express a manifold by a collection of local linear models, each depicted by a subspace. MMD is then converted to integrate the distances between pairs of subspaces from one of the involved manifolds. We theoretically and experimentally study several configurations of the ingredients of MMD. The proposed method is applied to the task of face recognition with image sets, where identification is achieved by seeking the minimum MMD from the probe to the gallery of image sets. Our experiments demonstrate that, as a general set similarity measure, MMD consistently outperforms other competing nondiscriminative methods and is also promisingly comparable to the state-of-the-art discriminative methods.