Matching 2.5D face scans to 3D models

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 28; no. 1; pp. 31 - 43
• Main Authors: Xiaoguang Lu, Jain, A.K., Colbry, D.
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.01.2006; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: 3D model; Algorithms; appearance-based; Applied sciences; Artificial Intelligence; Biometry - methods; Computer science; control theory; systems; Engines; Exact sciences and technology; Face - anatomy & histology; Face recognition; Galleries; Humans; Illumination; Image analysis; Image Interpretation, Computer-Assisted - methods; Image matching; Imaging, Three-Dimensional - methods; Independent component analysis; Index Terms- Face recognition; Iterative algorithms; Iterative closest point algorithm; Lighting; Matching; multimodal; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; Photogrammetry - methods; Photography - methods; Reproducibility of Results; Robustness; Sensitivity and Specificity; Shape; Studies; Subtraction Technique; Surface matching; Texture; Three dimensional; Image texture; Discriminant analysis; Face recognition; Image processing; Image databank; 3D model; multimodal; Pattern recognition; appearance-based; surface matching; Three dimensional model; Facies; Tridimensional image; Multimodal interface; Pattern analysis
• ISSN: 0162-8828; 1939-3539
• DOI: 10.1109/TPAMI.2006.15

The performance of face recognition systems that use two-dimensional images depends on factors such as lighting and subject's pose. We are developing a face recognition system that utilizes three-dimensional shape information to make the system more robust to arbitrary pose and lighting. For each subject, a 3D face model is constructed by integrating several 2.5D face scans which are captured from different views. 2.5D is a simplified 3D (x,y,z) surface representation that contains at most one depth value (z direction) for every point in the (x, y) plane. Two different modalities provided by the facial scan, namely, shape and texture, are utilized and integrated for face matching. The recognition engine consists of two components, surface matching and appearance-based matching. The surface matching component is based on a modified iterative closest point (ICP) algorithm. The candidate list from the gallery used for appearance matching is dynamically generated based on the output of the surface matching component, which reduces the complexity of the appearance-based matching stage. Three-dimensional models in the gallery are used to synthesize new appearance samples with pose and illumination variations and the synthesized face images are used in discriminant subspace analysis. The weighted sum rule is applied to combine the scores given by the two matching components. Experimental results are given for matching a database of 200 3D face models with 598 2.5D independent test scans acquired under different pose and some lighting and expression changes. These results show the feasibility of the proposed matching scheme.