HyperFace: A Deep Multi-Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 41; no. 1; pp. 121 - 135
• Main Authors: Ranjan, Rajeev, Patel, Vishal M., Chellappa, Rama
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Artificial neural networks; deep convolutional neural networks; Deep Learning; Face; Face - diagnostic imaging; Face detection; Face recognition; Feature extraction; Female; Fuses; Gender Identity; gender recognition; head pose estimation; Humans; Image Processing, Computer-Assisted - methods; Landmarks; landmarks localization; Localization; Machine learning; Male; multi-task learning; Pattern Recognition, Automated - methods; Pose estimation; Posture - physiology
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2017.2781233

We present an algorithm for simultaneous face detection, landmarks localization, pose estimation and gender recognition using deep convolutional neural networks (CNN). The proposed method called, HyperFace, fuses the intermediate layers of a deep CNN using a separate CNN followed by a multi-task learning algorithm that operates on the fused features. It exploits the synergy among the tasks which boosts up their individual performances. Additionally, we propose two variants of HyperFace: (1) HyperFace-ResNet that builds on the ResNet-101 model and achieves significant improvement in performance, and (2) Fast-HyperFace that uses a high recall fast face detector for generating region proposals to improve the speed of the algorithm. Extensive experiments show that the proposed models are able to capture both global and local information in faces and performs significantly better than many competitive algorithms for each of these four tasks.