Face Alignment via Regressing Local Binary Features

• Published in: IEEE transactions on image processing Vol. 25; no. 3; pp. 1233 - 1245
• Main Authors: Ren, Shaoqing, Cao, Xudong, Wei, Yichen, Sun, Jian
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.03.2016
• Subjects: Accuracy; Alignment; Biometric Identification - methods; Databases, Factual; Decision Trees; Detectors; Error detection; Face; Face - anatomy & histology; Face - diagnostic imaging; face alignment; Feature extraction; Humans; Image processing; Image Processing, Computer-Assisted - methods; Landmarks; local binary feature; random forest; Regression; Regression Analysis; Regression tree analysis; Shape; tracking; Training; Vegetation; random forest; Face alignment; tracking; local binary feature
• ISSN: 1057-7149; 1941-0042; 1941-0042
• DOI: 10.1109/TIP.2016.2518867

This paper presents a highly efficient and accurate regression approach for face alignment. Our approach has two novel components: 1) a set of local binary features and 2) a locality principle for learning those features. The locality principle guides us to learn a set of highly discriminative local binary features for each facial landmark independently. The obtained local binary features are used to jointly learn a linear regression for the final output. This approach achieves the state-of-the-art results when tested on the most challenging benchmarks to date. Furthermore, because extracting and regressing local binary features are computationally very cheap, our system is much faster than previous methods. It achieves over 3000 frames per second (FPS) on a desktop or 300 FPS on a mobile phone for locating a few dozens of landmarks. We also study a key issue that is important but has received little attention in the previous research, which is the face detector used to initialize alignment. We investigate several face detectors and perform quantitative evaluation on how they affect alignment accuracy. We find that an alignment friendly detector can further greatly boost the accuracy of our alignment method, reducing the error up to 16% relatively. To facilitate practical usage of face detection/alignment methods, we also propose a convenient metric to measure how good a detector is for alignment initialization.