A 3D graph convolutional networks model for 2D skeleton‐based human action recognition

• Published in: IET image processing Vol. 17; no. 3; pp. 773 - 783
• Main Authors: Weng, Libo, Lou, Weidong, Shen, Xin, Gao, Fei
• Format: Journal Article
• Language: English
• Published: Wiley 01.02.2023
• Subjects: 2D human action recognition; 3D convolutional neural networks; attention mechanism; graph convolutional neural networks; skeleton sequences
• ISSN: 1751-9659; 1751-9667
• DOI: 10.1049/ipr2.12671

With the popularity of cameras, the application of action recognition is more and more extensive. After the emergence of RGB‐D cameras and human pose estimation algorithms, human actions can be represented by a sequence of skeleton joints. Therefore, skeleton‐based action recognition has been a research hotspot. In this paper, a novel 3D Graph Convolutional Network model (3D‐GCN) with space‐time attention mechanism for 2D skeleton data is proposed. Three‐dimensional graph convolution is employed to extract spatiotemporal features of skeleton descriptor that is composed of joint coordinates, frame differences and angles. Meanwhile, different joints and different frames are given different attention to achieve action classification. A zebra crossing pedestrian dataset named ZCP is also provided, which simulates possible pedestrian actions on the zebra crossing in real scenes. Experimental evaluation is carried out on ZCP dataset and NTU RGB+D dataset. Experimental results show that our method is better than current 2D‐based methods and is comparable with 3D methods.