Toward unbiased emotion recognition: overcoming user bias with siamese convolutional networks

• Published in: Signal, image and video processing Vol. 19; no. 11
• Main Authors: La Porta, Nicolò, Oldano, Gilles, Puiatti, Alessandro, Leidi, Tiziano, Papandrea, Michela
• Format: Journal Article
• Language: English
• Published: London Springer London 01.11.2025; Springer Nature B.V
• Subjects: Bias; Computer Imaging; Computer Science; Datasets; Emotion recognition; Emotions; Human bias; Image Processing and Computer Vision; Machine learning; Multimedia Information Systems; Original Paper; Pattern Recognition and Graphics; Physiology; Qualitative analysis; Representations; Signal,Image and Speech Processing; Vision; Siamese Networks; Biosignal Processing; User Bias; Emotion Recognition; Affective Computing
• DOI: 10.1007/s11760-025-04500-1
• ISSN: 1863-1703

Purpose: Emotion Recognition (ER) systems are designed for an accurate inference of human emotions. Emotions are elicited by stimuli provided by the environment, such as social interactions or exposure to salient events, and are expressed by individuals in a strictly subjective manner. ER systems are usually based on machine learning approaches applied to physiological signals, which provide an authentic representation of emotions as they are not easily controlled or masked. However, ER remains a challenging task, due to a well-established user bias related to individual response specificity (IRS). When multiple subjects are considered together in a dataset, IRS makes the ER task more difficult at the dataset level than at the subject level. Methods: In this work, we accomplish a multi-class ER task based on physiological signals while addressing the user bias problem. We propose a mitigation strategy based on Siamese Convolutional Networks (SCN). We perform a qualitative and quantitative analysis of user bias, comparing the feature space of the proposed SCN with a baseline hand-crafted feature space. Results: Our SCN reached state-of-the-art performance in the considered multi-class ER task and showed great capability of disentangling user bias, by reducing the gap between subject-level and dataset-level metrics. Conclusion: Our main contribution is the proposal of a user bias mitigation strategy for ER tasks, which takes advantage of the SCN capability of producing subject-independent latent representations.