Infant AFAR: Automated facial action recognition in infants

• Published in: Behavior research methods Vol. 55; no. 3; pp. 1024 - 1035
• Main Authors: Onal Ertugrul, Itir, Ahn, Yeojin Amy, Bilalpur, Maneesh, Messinger, Daniel S., Speltz, Matthew L., Cohn, Jeffrey F.
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.04.2023; Springer Nature B.V
• Subjects: Adults; Automation; Babies; Behavioral Science and Psychology; Cognitive Psychology; Databases, Factual; Emotions; Face; Facial Expression; Facial Recognition; Humans; Infant; Infants; Neural networks; Neural Networks, Computer; Pattern recognition; Psychology; Sensors; Social Interaction; Deep learning; Cross domain generalizability; Automatic facial action unit detection; Infant behavior; Facial action coding system
• ISSN: 1554-3528; 1554-351X; 1554-3528
• DOI: 10.3758/s13428-022-01863-y

Automated detection of facial action units in infants is challenging. Infant faces have different proportions, less texture, fewer wrinkles and furrows, and unique facial actions relative to adults. For these and related reasons, action unit (AU) detectors that are trained on adult faces may generalize poorly to infant faces. To train and test AU detectors for infant faces, we trained convolutional neural networks (CNN) in adult video databases and fine-tuned these networks in two large, manually annotated, infant video databases that differ in context, head pose, illumination, video resolution, and infant age. AUs were those central to expression of positive and negative emotion. AU detectors trained in infants greatly outperformed ones trained previously in adults. Training AU detectors across infant databases afforded greater robustness to between-database differences than did training database specific AU detectors and outperformed previous state-of-the-art in infant AU detection. The resulting AU detection system, which we refer to as Infant AFAR (Automated Facial Action Recognition), is available to the research community for further testing and applications in infant emotion, social interaction, and related topics.