Improved human action recognition approach based on two-stream convolutional neural network model

• Published in: The Visual computer Vol. 37; no. 6; pp. 1327 - 1341
• Main Authors: Liu, Congcong, Ying, Jie, Yang, Haima, Hu, Xing, Liu, Jin
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.06.2021; Springer Nature B.V
• Subjects: Algorithms; Annotations; Artificial Intelligence; Artificial neural networks; Computer Graphics; Computer Science; Deep learning; Human activity recognition; Image Processing and Computer Vision; Kalman filters; Neural networks; Original Article; Performance evaluation; Recognition; Spacetime; Support vector machines; Training; Wavelet transforms; Human action recognition; Video surveillance; Motion history image; Faster R-CNN; Kalman filter
• ISSN: 0178-2789; 1432-2315
• DOI: 10.1007/s00371-020-01868-8

In order to improve the accuracy of human abnormal behavior recognition, a two-stream convolution neural network model was proposed. This model includes two main parts, VMHI and FRGB. Firstly, the motion history images are extracted and input into VGG-16 convolutional neural network for training. Then, the RGB image is input into Faster R-CNN algorithm for training using Kalman filter-assisted data annotation. Finally, the two stream VMHI and FRGB results are fused. The algorithm can recognize not only single person behavior, but also two person interaction behavior and improve the recognition accuracy of similar actions. Experimental results on KTH, Weizmann, UT-interaction, and TenthLab dataset showed that the proposed algorithm has higher accuracy than the other literature.