Unbiased Decision-Making Framework in Long-Video Macro & Micro-Expression Spotting

• Published in: 2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) pp. 84 - 89
• Main Authors: Tan, Pei-Sze, Rajanala, Sailaja, Pal, Arghya, Phan, Raphael C.-W., Ong, Huey-Fang
• Format: Conference Proceeding
• Language: English
• Published: IEEE 31.10.2023
• Subjects: Computational modeling; Decision making; Deep learning; Graphical models; Information processing; Training; Training data
• ISSN: 2640-0103
• DOI: 10.1109/APSIPAASC58517.2023.10317342

Spotting macro and micro expressions proves more demanding than subsequent tasks such as recognizing or classifying expressions. This intricate process entails pinpointing the precise commencement, conclusion, and pinnacle frames of brief facial expressions within video footage. Our study delves into the susceptibility of expression spotting to biases ingrained in deep learning models, often stemming from a lack of diversity in the training data. These biases manifest diversely, such as an overemphasis on specific expression categories, genders, or racial backgrounds, leading to subpar performance on other samples. These spurious biases in deep learning models adversely affect classification accuracy. Our work aims to unearth common biases in micro and macro expression spotting and suggests methods to detect them. In addressing this challenge, we introduce a causal graph that elucidates the connections among a trained deep learning model, the feature space it acquires, and the resulting outcomes. This causal graph plays a pivotal role in pinpointing potential origins of bias during the training phase. We utilise counterfactual inputs to evaluate model biases, providing advantages in terms of computational efficiency and interpretability when contrasted with in-training debiasing approaches. This makes counterfactual debiasing a promising avenue for addressing biases in machine learning models. Experimental results underscore the effectiveness of this approach in successfully mitigating biases in several state-of-the-art macro and micro-expression spotting methods.