Facial expression recognition of intercepted video sequences based on feature point movement trend and feature block texture variation

• Published in: Applied soft computing Vol. 82; p. 105540
• Main Authors: Yi, Jizheng, Chen, Aibin, Cai, Zixing, Sima, Yi, Zhou, Mengna, Wu, Xingyu
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.09.2019
• Subjects: Facial expression recognition (FER); Facial expression sequence interception (FESI); Feature block texture difference (FBTD); Slope set; Facial expression recognition (FER); Feature block texture difference (FBTD); Facial expression sequence interception (FESI); Slope set
• ISSN: 1568-4946; 1872-9681
• DOI: 10.1016/j.asoc.2019.105540

Facial Expression Recognition (FER) is an important subject of human–computer interaction and has long been a research area of great interest. Accurate Facial Expression Sequence Interception (FESI) and discriminative expression feature extraction are two enormous challenges for the video-based FER. This paper proposes a framework of FER for the intercepted video sequences by using feature point movement trend and feature block texture variation. Firstly, the feature points are marked by Active Appearance Model (AAM) and the most representative 24 of them are selected. Secondly, facial expression sequence is intercepted from the face video by determining two key frames whose emotional intensities are minimum and maximum, respectively. Thirdly, the trend curve which represents the Euclidean distance variations between any two selected feature points is fitted, and the slopes of specific points on the trend curve are calculated. Finally, combining Slope Set which is composed by the calculated slopes with the proposed Feature Block Texture Difference (FBTD) which refers to the texture variation of facial patch, the final expressional feature are formed and inputted to One-dimensional Convolution Neural Network (1DCNN) for FER. Five experiments are conducted in this research, and three average FER rates 95.2%, 96.5%, and 97% for Beihang University (BHU) facial expression database, MMI facial expression database, and the combination of two databases, respectively, have shown the significant advantages of the proposed method over the existing ones. •The proposed method can replace the original manual interception.•The Slope Set from the fitted trend curve can express feature points’ movement trend.•The dynamic slope information are insensitive to the individual differences.•Feature Block Texture Difference can extract the discriminative appearance features.