Recognizing Emotion Regulation Strategies from Human Behavior with Large Language Models

• Published in: International Conference on Affective Computing and Intelligent Interaction and workshops pp. 210 - 218
• Main Authors: Muller, Philipp, Heimerl, Alexander, Hossain, Sayed Muddashir, Siegel, Lea, Alexandersson, Jan, Gebhard, Patrick, Andre, Elisabeth, Schneeberger, Tanja
• Format: Conference Proceeding
• Language: English
• Published: IEEE 15.09.2024
• Subjects: Affective computing; Bayes methods; bayesian networks; Behavioral sciences; Emotion recognition; emotion regulation; Interviews; Large language models; Optimization; Regulation; Sentiment analysis; Training
• ISSN: 2156-8111
• DOI: 10.1109/ACII63134.2024.00029

Human emotions are often not expressed directly, but regulated according to internal processes and social display rules. For affective computing systems, an understanding of how users regulate their emotions can be highly useful, for example to provide feedback in job interview training, or in psychotherapeutic scenarios. However, at present no method to automatically classify different emotion regulation strategies in a cross-user scenario exists. At the same time, recent studies showed that instruction-tuned Large Language Models (LLMs) can reach impressive performance across a variety of affect recognition tasks such as categorical emotion recognition or sentiment analysis. While these results are promising, it remains unclear to what extent the representational power of LLMs can be utilized in the more subtle task of classifying users' internal emotion regulation strategy. To close this gap, we make use of the recently introduced Deep corpus for modeling the social display of the emotion shame, where each point in time is annotated with one of seven different emotion regulation classes. We fine-tune Llama2-7B as well as the recently introduced Gemma model using Low-rank Optimization on prompts generated from different sources of information on the Deep corpus. These include verbal and nonverbal behavior, person factors, as well as the results of an indepth interview after the interaction. Our results show, that a fine-tuned Llama2-7B LLM is able to classify the utilized emotion regulation strategy with high accuracy (0.84) without needing access to data from post-interaction interviews. This represents a significant improvement over previous approaches based on Bayesian Networks and highlights the importance of modeling verbal behavior in emotion regulation.