Skeleton-based Human Action Recognition via Large-kernel Attention Graph Convolutional Network

• Published in: IEEE transactions on visualization and computer graphics Vol. 29; no. 5; pp. 2575 - 2585
• Main Authors: Liu, Yanan, Zhang, Hao, Li, Yanqiu, He, Kangjian, Xu, Dan
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.05.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: action recognition; Adaptation models; Artificial neural networks; Background noise; Computer architecture; Convolution; graph convolution; Graphical representations; Human activity recognition; human skeleton; Joints; Kernel; Kernels; large kernels; Modelling; Skeleton; Task analysis; Topology; Virtual reality
• ISSN: 1077-2626; 1941-0506; 1941-0506
• DOI: 10.1109/TVCG.2023.3247075

The skeleton-based human action recognition has broad application prospects in the field of virtual reality, as skeleton data is more resistant to data noise such as background interference and camera angle changes. Notably, recent works treat the human skeleton as a non-grid representation, e.g., skeleton graph, then learns the spatio-temporal pattern via graph convolution operators. Still, the stacked graph convolution plays a marginal role in modeling long-range dependences that may contain crucial action semantic cues. In this work, we introduce a skeleton large kernel attention operator (SLKA), which can enlarge the receptive field and improve channel adaptability without increasing too much computational burden. Then a spatiotemporal SLKA module (ST-SLKA) is integrated, which can aggregate long-range spatial features and learn long-distance temporal correlations. Further, we have designed a novel skeleton-based action recognition network architecture called the spatiotemporal large-kernel attention graph convolution network (LKA-GCN). In addition, large-movement frames may carry significant action information. This work proposes a joint movement modeling strategy (JMM) to focus on valuable temporal interactions. Ultimately, on the NTU-RGBD 60, NTU-RGBD 120 and Kinetics-Skeleton 400 action datasets, the performance of our LKA-GCN has achieved a state-of-the-art level.