Micro-Expression-Based Emotion Recognition Using Waterfall Atrous Spatial Pyramid Pooling Networks

• Published in: Sensors (Basel, Switzerland) Vol. 22; no. 12; p. 4634
• Main Authors: Stofa, Marzuraikah Mohd, Zulkifley, Mohd Asyraf, Zainuri, Muhammad Ammirrul Atiqi Mohd
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 19.06.2022; MDPI
• Subjects: Automation; Classification; convolutional neural networks; Datasets; deep learning; emotion classification; Emotions; Machine learning; micro-expression analysis; Network management systems; Semantics; Support vector machines
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s22124634

Understanding a person’s attitude or sentiment from their facial expressions has long been a straightforward task for humans. Numerous methods and techniques have been used to classify and interpret human emotions that are commonly communicated through facial expressions, with either macro- or micro-expressions. However, performing this task using computer-based techniques or algorithms has been proven to be extremely difficult, whereby it is a time-consuming task to annotate it manually. Compared to macro-expressions, micro-expressions manifest the real emotional cues of a human, which they try to suppress and hide. Different methods and algorithms for recognizing emotions using micro-expressions are examined in this research, and the results are presented in a comparative approach. The proposed technique is based on a multi-scale deep learning approach that aims to extract facial cues of various subjects under various conditions. Then, two popular multi-scale approaches are explored, Spatial Pyramid Pooling (SPP) and Atrous Spatial Pyramid Pooling (ASPP), which are then optimized to suit the purpose of emotion recognition using micro-expression cues. There are four new architectures introduced in this paper based on multi-layer multi-scale convolutional networks using both direct and waterfall network flows. The experimental results show that the ASPP module with waterfall network flow, which we coined as WASPP-Net, outperforms the state-of-the-art benchmark techniques with an accuracy of 80.5%. For future work, a high-resolution approach to multi-scale approaches can be explored to further improve the recognition performance.