Micro-expression spotting network based on attention and one-dimensional convolutional sliding window

• Published in: Multimedia systems Vol. 29; no. 4; pp. 2429 - 2437
• Main Authors: Xing, Hongbo, Zhou, Guanqun, Yuan, Shusen, Jiang, Youjun, Geng, Pinyong, Cao, Yewen, Li, Yujun, Chen, Lei
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.08.2023; Springer Nature B.V
• Subjects: Algorithms; Artificial neural networks; Computer Communication Networks; Computer Graphics; Computer Science; Computer vision; Cryptology; Data Storage Representation; Intervals; Multimedia Information Systems; Operating Systems; Regular Paper; Sliding; Video data; Deep learning; Attention mechanism; Long videos; Micro-expression spotting
• ISSN: 0942-4962; 1432-1882
• DOI: 10.1007/s00530-023-01120-y

In the field of computer vision, the research on micro-expression (ME) can be divided into two main tasks: ME spotting and ME recognition. ME spotting refers to finding the occurrence and interval of ME from video stream, which is an indispensable module for automatic ME analysis. In this paper, aiming at finding of the inaccurate location of MEs in long videos, we proposed a ME spotting network based on attention mechanism and one-dimensional (1D) convolution sliding window. In our proposed scheme, convolutional neural network (CNN), Bi-directional Long Short-Term Memory (BI-LSTM), and 1D convolution are used to extract features. The attention mechanism is used to highlight the key frames. 1D convolution with sliding window is applied to detect feature intervals, which are further combined with the intervals and judged as MEs to obtain the final ME spotting result. Simulation was done on CAS(ME)2 dataset. It is shown that the proposed algorithm outperforms other superior algorithms in terms of effectiveness.