Micro-expression recognition based on Spatio-temporal capsule network

• Published in: IEEE access Vol. 11; p. 1
• Main Authors: Shang, Ziyang, Liu, Jie, Li, Xinfu
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Algorithms; Capsule network; Data mining; deep learning; differential features; Face recognition; Feature extraction; Heuristic algorithms; Machine learning; micro-expression recognition; Modules; Optical computing; Optical distortion; Optical imaging; Psychological stress; Recognition; Redundancy; spatio-temporal features; spatiotemporal features; Vibration
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2023.3242871

Micro-expression (ME) is a subtle and spontaneous natural facial mechanism that is one of the essential psychological stress responses. Due to its accuracy and uncontrollability for psychological expressions, ME has crucial applications in many psychologically related fields. However, due to small data volume and high data redundancy, existing micro-expression recognition (MER) methods cannot balance accuracy and recognition speed. Therefore, we propose a deep learning algorithm based on a spatio-temporal capsule network (STCP-Net). It reduces recognition time while ensuring the accuracy rate. STCP-Net consists of four parts: a jitter removal module, a differential feature extraction module, a spatio-temporal capsule (STCP) module, and a fully connected layer. The first two modules extract diversifying differential features more accurately and reduce the impact of head jitter. The STCP module extracts spatio-temporal features progressively layer by layer, fully considering the temporal and spatial relationship between features. Finally, this study conducts sufficient experiments based on the Leave One Subject Out (LOSO) cross-validation protocol with standard datasets. The final results and analysis show that the algorithm is advanced and effective.