Compound facial expressions of emotion

• Published in: Proceedings of the National Academy of Sciences - PNAS Vol. 111; no. 15; pp. E1454 - E1462
• Main Authors: Du, Shichuan, Tao, Yong, Martinez, Aleix M.
• Format: Journal Article
• Language: English
• Published: United States National Academy of Sciences 15.04.2014; National Acad Sciences
• Series: PNAS Plus
• Subjects: Adult; Biological Sciences; cognition; Cognition & reasoning; Discrimination (Psychology) - physiology; Emotions; Emotions - classification; Emotions - physiology; Face; Facial Expression; Facial Muscles - physiology; fearfulness; Female; Humans; Male; Models, Biological; muscles; Ohio; Photography; Physical Sciences; PNAS Plus; Sensory perception; Ohio; categorization; face recognition; action units
• ISSN: 0027-8424; 1091-6490; 1091-6490
• DOI: 10.1073/pnas.1322355111

Understanding the different categories of facial expressions of emotion regularly used by us is essential to gain insights into human cognition and affect as well as for the design of computational models and perceptual interfaces. Past research on facial expressions of emotion has focused on the study of six basic categories—happiness, surprise, anger, sadness, fear, and disgust. However, many more facial expressions of emotion exist and are used regularly by humans. This paper describes an important group of expressions, which we call compound emotion categories. Compound emotions are those that can be constructed by combining basic component categories to create new ones. For instance, happily surprised and angrily surprised are two distinct compound emotion categories. The present work defines 21 distinct emotion categories. Sample images of their facial expressions were collected from 230 human subjects. A Facial Action Coding System analysis shows the production of these 21 categories is different but consistent with the subordinate categories they represent (e.g., a happily surprised expression combines muscle movements observed in happiness and surprised). We show that these differences are sufficient to distinguish between the 21 defined categories. We then use a computational model of face perception to demonstrate that most of these categories are also visually discriminable from one another.