3D-aware Facial Landmark Detection via Multi-view Consistent Training on Synthetic Data

Accurate facial landmark detection on wild images plays an essential role in human-computer interaction, entertainment, and medical applications. Existing approaches have limitations in enforcing 3D consistency while detecting 3D/2D facial landmarks due to the lack of multi-view in-the-wild training...

Full description

Saved in:
Bibliographic Details
Published in2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 12747 - 12758
Main Authors Zeng, Libing, Chen, Lele, Bao, Wentao, Li, Zhong, Xu, Yi, Yuan, Junsong, Kalantari, Nima K.
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2023
Subjects
body
gesture
Humans: Face
movement
Pipelines
pose
Pose estimation
Solid modeling
Three-dimensional displays
Training
Training data
Visualization
Online AccessGet full text

Cover

More Information
Summary:Accurate facial landmark detection on wild images plays an essential role in human-computer interaction, entertainment, and medical applications. Existing approaches have limitations in enforcing 3D consistency while detecting 3D/2D facial landmarks due to the lack of multi-view in-the-wild training data. Fortunately, with the recent advances in generative visual models and neural rendering, we have witnessed rapid progress towards high quality 3D image synthesis. In this work, we leverage such approaches to construct a synthetic dataset and propose a novel multi-view consistent learning strategy to improve 3D facial landmark detection accuracy on in-the-wild images. The proposed 3D-aware module can be plugged into any learning-based landmark detection algorithm to enhance its accuracy. We demonstrate the superiority of the proposed plug-in module with extensive comparison against state-of-the-art methods on several real and synthetic datasets.
ISSN:2575-7075
DOI:10.1109/CVPR52729.2023.01226