Dual complementarity transformer for micro-expression recognition

• Published in: Multimedia systems Vol. 31; no. 5
• Main Authors: Zhou, Ying, Chen, Lei, Huang, Tianhuan, Liu, Ju, Ben, Xianye
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.10.2025; Springer Nature B.V
• Subjects: Computer Communication Networks; Computer Graphics; Computer Science; Cryptology; Data Storage Representation; Datasets; Emotions; Face recognition; Feature extraction; Multimedia Information Systems; Operating Systems; Regular Paper; Complementary strategy; Micro-expression recognition; Transformer; Identity information
• ISSN: 0942-4962; 1432-1882
• DOI: 10.1007/s00530-025-01960-w

Facial micro-expressions (MEs) are transient and spontaneous facial muscle motions which can reveal the genuine emotions that people strive to conceal. However, in facial expression recognition, the coupling of identity and emotion information can impact performance, especially in micro-expression recognition (MER). Moreover, due to the inherent properties of MEs, extracting robust and discriminative features to enhance recognition performance is a challenge. To address these issues, a novel Transformer-based architecture called the Dual Complementarity Transformer (DCT) is proposed. Specifically, to reduce the interference of identity information, a motion manipulator is designed to capture and amplify the emotional features. Then, the amplified emotional features are further enhanced through a complementary strategy which effectively captures global features by alternately computing spatial and channel self-attention. Subsequently, a convolutional representation enhancement module is designed, which aims to equip the DCT with the locality. Therefore, the proposed method achieves discriminative feature extraction by complementarily aggregating multiple types of features (spatial-channel, global–local), and significantly improves MER performance. Extensive experiments conducted under the Leave-One-Subject-Out protocol on widely adopted datasets, including SMIC, CASME II, SAMM, and MEGC 2019 composite dataset, demonstrate that the proposed DCT outperforms other related MER approaches.