RES-CapsNet: an improved capsule network for micro-expression recognition - Shu, Xin; Li, Jia; Shi, Liang; Huang, Shucheng (Multimedia systems, 2023)
Micro-expression is a type of facial expression that reveals the deepest feeling held within the human heart. Despite the substantial improvement that has been achieved, micro-expression recognition remains a significant challenge considering its low intensity and short duration. In this paper, we investigate the recognition of micro-expression using deep learning techniques and present the RES-CapsNet, which is an improved capsule network that employs Res2Net as the backbone to extract multi-level and multi-scale characteristics. Furthermore, RES-CapsNet adds a squeeze-excitation (SE) block to the primary capsule layer (PrimaryCaps). Benefiting from a SE block, the valuable micro-expression features are highlighted and the useless ones are suppressed. In addition, between the first convolutional layer and the PrimaryCaps in RES-CapsNet, we introduce an effective channel attention (ECA) module that simply includes a few parameters while dramatically improving the performance. The proposed architecture initially obtains apex frames from the micro-expression sequence to capture the most distinct facial muscle movements and then feeds the pre-processed images into RES-CapsNet for further feature extraction and classification. The Leave-One-Subject-Out (LOSO) cross-validation strategy is implemented on three prevalent spontaneous micro-expression databases (i.e., CASME II, SMIC, and SAMM) to assess the feasibility of our RES-CapsNet. Extensive experiments demonstrate that our RES-CapsNet describes considerable details of micro-expression effectively and achieves superiorly higher performance than the baseline CapsuleNet.