Face pose estimation with automatic 3D model creation in challenging scenarios

• Published in: Image and vision computing Vol. 30; no. 9; pp. 589 - 602
• Main Authors: Jiménez, Pedro, Bergasa, Luis M., Nuevo, Jesús, Alcantarilla, Pablo F.
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.09.2012
• Subjects: 3D face pose estimation; Algorithms; Cameras; Computer simulation; Face model; Feature re-registering; Run time (computers); Stereo vision; Three dimensional; Three dimensional models; Yaw; Yaw rotation; Yaw rotation; Stereo vision; Feature re-registering; Face model; 3D face pose estimation
• ISSN: 0262-8856; 1872-8138
• DOI: 10.1016/j.imavis.2012.06.013

This paper proposes a new method to perform real-time face pose estimation for ±90° yaw rotations and under low light conditions. The algorithm works on the basis of a completely automatic and run-time incremental 3D face modelling. The model is initially made up upon a set of 3D points derived from stereo grey-scale images. As new areas of the subject face appear to the cameras, new 3D points are automatically added to complete the model. In this way, we can estimate the pose for a wide range of rotation angles, where typically 3D frontal points are occluded. We propose a new feature re-registering technique which combines views of both cameras of the stereo rig in a smart way in order to perform a fast and robust tracking for the full range of yaw rotations. The Levenberg–Marquardt algorithm is used to recover the pose and a RANSAC framework rejects incorrectly tracked points. The model is continuously optimised in a bundle adjustment process that reduces the accumulated error on the 3D reconstruction. The intended application of this work is estimating the focus of attention of drivers in a simulator, which imposes challenging requirements. We validate our method on sequences recorded in a naturalistic truck simulator, on driving exercises designed by a team of psychologists. [Display omitted] ► We develop a 3D face pose estimation algorithm to work on low lighting conditions. ► We start by creating a 3D model of the face. ► Feature templates are updated dynamically. ► The model is extended and corrected online. ► Results from a naturalistic driving simulator are compared to a ground-truth.