Microexpression Identification and Categorization Using a Facial Dynamics Map

• Published in: IEEE transactions on affective computing Vol. 8; no. 2; pp. 254 - 267
• Main Authors: Feng Xu, Junping Zhang, Wang, James Z.
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.04.2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptive optics; Cameras; Dynamics; emotions; expression recognition; Face; Face recognition; Frequency division multiplexing; Identification methods; Microexpression; optical flow; Optical flow (image analysis); Optical imaging; Strain; Tagging
• ISSN: 1949-3045; 1949-3045
• DOI: 10.1109/TAFFC.2016.2518162

Unlike conventional facial expressions, microexpressions are instantaneous and involuntary reflections of human emotion. Because microexpressions are fleeting, lasting only a few frames within a video sequence, they are difficult to perceive and interpret correctly, and they are highly challenging to identify and categorize automatically. Existing recognition methods are often ineffective at handling subtle face displacements, which can be prevalent in typical microexpression applications due to the constant movements of the individuals being observed. To address this problem, a novel method called the Facial Dynamics Map is proposed to characterize the movements of a microexpression in different granularity. Specifically, an algorithm based on optical flow estimation is used to perform pixel-level alignment for microexpression sequences. Each expression sequence is then divided into spatiotemporal cuboids in the chosen granularity. We also present an iterative optimal strategy to calculate the principal optical flow direction of each cuboid for better representation of the local facial dynamics. With these principal directions, the resulting Facial Dynamics Map can characterize a microexpression sequence. Finally, a classifier is developed to identify the presence of microexpressions and to categorize different types. Experimental results on four benchmark datasets demonstrate higher recognition performance and improved interpretability.