AU‐Guided Feature Aggregation for Micro‐Expression Recognition

• Published in: Computer animation and virtual worlds Vol. 36; no. 3
• Main Authors: Tan, Xiaohui, Xu, Weiqi, Wu, Jiazheng, Geng, Hao, Geng, Qichuan
• Format: Journal Article
• Language: English
• Published: Hoboken, USA John Wiley & Sons, Inc 01.05.2025; Wiley Subscription Services, Inc
• Subjects: Ablation; action unit; Deep learning; high‐level relation; micro expression recognition; Representations; spatial‐temporal; Statistical analysis
• DOI: 10.1002/cav.70041
• ISSN: 1546-4261

ABSTRACT Micro‐expressions (MEs) are spontaneous and transient facial movements that reflect real internal emotions and have been widely applied in various fields. Recent deep learning‐based methods have been rapidly developing in micro‐expression recognition (MER).Still, it is typical to focus on the one‐sided nature of MEs, covering only representational features or low‐ranking Action Unit (AU) features. The subtle changes in MEs characterize its feature representation weak and inconspicuous, making it tough to analyze MEs only from a single piece or a small amount of information to achieve a considerable recognition effect. In addition, the lower‐order information can only distinguish MEs from a single low‐dimensional perspective and neglects the potential of corresponding MEs and AU combinations to each other. To address these issues, we first explore how the higher‐order relations of different AU combinations correspond with MEs through statistical analysis. Afterward, based on this attribute, we propose an end‐to‐end multi‐stream model that integrates global feature learning and local muscle movement representation guided by AU semantic information. The comparative experiments were performed on benchmark datasets, with better performance than the state‐of‐art methods. Also, the ablation experiments demonstrate the necessity of our model to introduce the information of AU and its relationship to MER.