Robust Facial Feature Tracking Using Shape-Constrained Multiresolution-Selected Linear Predictors

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 33; no. 9; pp. 1844 - 1859
• Main Authors: Eng-Jon Ong, Bowden, Richard
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.09.2011; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Artificial intelligence; Biometric Identification - methods; Computer science; control theory; systems; Databases, Factual; Exact sciences and technology; Face - anatomy & histology; Facial; Facial feature tracking; Facial features; Failure; Feature extraction; Humans; Image Processing, Computer-Assisted - methods; Intelligence; learning; linear predictors; Linear programming; Mathematical models; Models, Statistical; Mouth; multiple resolution; Pattern analysis; Pattern recognition. Digital image processing. Computational geometry; probabilistic selection; Studies; Target tracking; Tracking; Visual; Visualization; probabilistic selection; Tracking; Image processing; linear predictors; Linear group; learning; Data driven modelling; Modeling; Multiresolution analysis; Deformable body; Linear prediction; Visual cue; Robustness; Computer vision; Probabilistic approach; Motion estimation; Point shape; Face recognition; multiple resolution; Large displacement; Real time; Facial feature tracking; Visual stimulus; A priori estimation; Hierarchical system; Biased estimation
• ISSN: 0162-8828; 1939-3539; 2160-9292; 1939-3539
• DOI: 10.1109/TPAMI.2010.205

This paper proposes a learned data-driven approach for accurate, real-time tracking of facial features using only intensity information. The task of automatic facial feature tracking is nontrivial since the face is a highly deformable object with large textural variations and motion in certain regions. Existing works attempt to address these problems by either limiting themselves to tracking feature points with strong and unique visual cues (e.g., mouth and eye corners) or by incorporating a priori information that needs to be manually designed (e.g., selecting points for a shape model). The framework proposed here largely avoids the need for such restrictions by automatically identifying the optimal visual support required for tracking a single facial feature point. This automatic identification of the visual context required for tracking allows the proposed method to potentially track any point on the face. Tracking is achieved via linear predictors which provide a fast and effective method for mapping pixel intensities into tracked feature position displacements. Building upon the simplicity and strengths of linear predictors, a more robust biased linear predictor is introduced. Multiple linear predictors are then grouped into a rigid flock to further increase robustness. To improve tracking accuracy, a novel probabilistic selection method is used to identify relevant visual areas for tracking a feature point. These selected flocks are then combined into a hierarchical multiresolution LP model. Finally, we also exploit a simple shape constraint for correcting the occasional tracking failure of a minority of feature points. Experimental results show that this method performs more robustly and accurately than AAMs, with minimal training examples on example sequences that range from SD quality to Youtube quality. Additionally, an analysis of the visual support consistency across different subjects is also provided.