Modeling the 'Kiss my Ass' -Smile: Appearance and Functions of Smiles in Negative Social Situations

Computational emotion recognition relies on observable expressions. However, negative situations can evoke regulation mechanisms that obscure and mask emotional experiences, often by smiling. As smiles are typically associated with positive emotions, this mismatch of emotional experience and express...

Full description

Saved in:
Bibliographic Details
Published inInternational Conference on Affective Computing and Intelligent Interaction and workshops pp. 116 - 124
Main Authors Hladky, Mirella, Guerra, Rubia R., Cang, X. Laura, MacLean, Karon E., Gebhard, Patrick, Schneeberger, Tanja
Format Conference Proceeding
LanguageEnglish
Published IEEE 15.09.2024
Subjects
Affective computing
Benchmark testing
Computational modeling
Corpus
Data models
Emotion Modeling
Emotion recognition
Encoding
Face recognition
Facial Coding
Morphology
Recording
Regulation
Online AccessGet full text

Cover

More Information
Summary:Computational emotion recognition relies on observable expressions. However, negative situations can evoke regulation mechanisms that obscure and mask emotional experiences, often by smiling. As smiles are typically associated with positive emotions, this mismatch of emotional experience and expression may lead to misinterpretations by most current algorithmic affective computing approaches. To improve computational modeling of real-life experiences and expressions in negative social situations, we explore connections between smile appearance and function, incorporating participants' rich personal self-reports into ground truth labels for their expressions. We present an empirically grounded smile corpus of 199 smiles that is based on a) recordings of N = 30 participants in negative social situations that are analyzed regarding smile morphology and b) a category system of smile functions based on participants' self-reports. In a computational model, we used cleaned corpus data of 183 unique smile instances to classify five smile function categories based on observable nonverbal signals, with results benchmarked at above chance. Applying a theory- and data-driven approach, our analyses confirm a complex relationship between internal smile functions and observable signals. Finally, we discuss smile functions in negative social situations, including 'despising', 'provoking', and 'kiss my ass'-smiles.
ISSN:2156-8111
DOI:10.1109/ACII63134.2024.00018