Synthesis of Facial Expressions in Photographs Characteristics, Approaches, and Challenges

• Published in: ACM computing surveys Vol. 51; no. 6; pp. 1 - 35
• Main Authors: Testa, Rafael Luiz, Corrêa, Cléber Gimenez, Machado-Lima, Ariane, Nunes, Fátima L. S.
• Format: Journal Article
• Language: English
• Published: New York, NY, USA ACM 30.11.2019; Association for Computing Machinery
• Subjects: Biometrics; Computer graphics; Computer science; Computing methodologies; Deformation; Image manipulation; Image-based rendering; Interactive systems; Machine learning; Mapping; Synthesis; Training; facial expression generation; facial expression mapping; expression transfer; Facial expression synthesis; facial expression cloning
• ISSN: 0360-0300; 1557-7341
• DOI: 10.1145/3292652

The synthesis of facial expressions has applicationsin areas such as interactive games, biometrics systems, and training of people with disorders, among others. Although this is an area relatively well explored in the literature, there are no recent studies proposing to systematize an overview of research in the area. This systematic review analyzes the approaches to the synthesis of facial expressions in photographs, as well as important aspects of the synthesis process, such as preprocessing techniques, databases, and evaluation metrics. Forty-eight studies from three different scientific databases were analyzed. From these studies, we established an overview of the process, including all the stages used to synthesize expressions in facial images. We also analyze important aspects involved in these stages such as methods and techniques of each stage, databases, and evaluation metrics. We observed that machine learning approaches are the most widely used to synthesize expressions. Landmark identification, deformation, mapping, fusion, and training are common tasks considered in the approaches. We also found that few studies used metrics to evaluate the results, and most studies used public databases. Although the studies analyzed generated consistent and realistic results while preserving the identity of the subject, there are still research themes to be exploited.