FSAGN:An expression recognition method based on independent selection of video key frames

• Published in: Zhejiang da xue xue bao. Journal of Zhejiang University. Sciences edition. Li xue ban Vol. 49; no. 2; pp. 141 - 150
• Main Authors: Zhu, Jintai, Ye, Jihua, Guo, Feng, Jiang, Lu, Jiang, Aiwen
• Format: Journal Article
• Language: Chinese
• Published: Hangzhou Zhejiang University 01.03.2022; Zhejiang University Press
• Subjects: Datasets; Face recognition; Feature extraction; Frames (data processing); ghostnet; Mathematical models; Parameters; Training; Video data; 关键帧自主选择; 注意力机制; 面部表情识别
• ISSN: 1008-9497
• DOI: 10.3785/j.issn.1008-9497.2022.02.002

As there exist a large number of video frames unrelated to facial expressions in the video data set containing facial expressions, a large amount of information unrelated to facial expressions is learned in the training process of the model, which results in a significant decline of the performance. So how to make the model capable of choosing the relevant video key frame autonomously becomes the key problem. At present, most of the existing video expression recognition methods do not yet consider the different effects of key frame and non-key frame on the training effect of the model. In the paper, a face expression recognition model based on attention mechanism and GhostNet(FSAGN) is proposed. The model calculates the weights of different frames by self-attention mechanism and frame selection loss, then selects the key frames of the video sequence autonomously according to the weights. In addition, in order to reduce model parameters and training costs, our approach replaces the traditional feature extracti