Emotion Recognition in Simulated Social Interactions

• Published in: IEEE transactions on affective computing Vol. 11; no. 2; pp. 308 - 312
• Main Authors: Mumenthaler, C., Sander, D., Manstead, A. S. R.
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.04.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Avatars; cognitive appraisal; Cognitive processes; Computer simulation; Emotion recognition; Emotions; Eye movements; Face recognition; facial expressions; Head movement; Mixtures; Social factors; social inferences; Social interaction
• ISSN: 1949-3045; 1949-3045
• DOI: 10.1109/TAFFC.2018.2799593

Social context plays an important role in everyday emotional interactions, and others' faces often provide contextual cues in social situations. Investigating this complex social process is a challenge that can be addressed with the use of computer-generated facial expressions. In the current research, we use synthesized facial expressions to investigate the influence of socioaffective inferential mechanisms on the recognition of social emotions. Participants judged blends of facial expressions of shame-sadness, or of anger-disgust, in a target avatar face presented at the center of a screen while a contextual avatar face expressed an emotion (disgust, contempt, and sadness) or remained neutral. The dynamics of the facial expressions and the head/gaze movements of the two avatars were manipulated in order to create an interaction in which the two avatars shared eye gaze only in the social interaction condition. Results of Experiment 1 revealed that when the avatars engaged in social interaction, target expression blends of shame and sadness were perceived as expressing more shame if the contextual face expressed disgust and more sadness when the contextual face expressed sadness. Interestingly, perceptions of shame were not enhanced when the contextual face expressed contempt. The latter finding is probably attributable to the low recognition rates for the expression of contempt observed in Experiment 2.