Action Recognition Based on Multi-Level Topological Channel Attention of Human Skeleton

• Published in: Sensors (Basel, Switzerland) Vol. 23; no. 24; p. 9738
• Main Authors: Hu, Kai, Shen, Chaowen, Wang, Tianyan, Shen, Shuai, Cai, Chengxue, Huang, Huaming, Xia, Min
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 10.12.2023; MDPI
• Subjects: Algorithms; Computational linguistics; Deep learning; Extremities; Human mechanics; Humans; Knowledge; Language processing; Learning; Natural language interfaces; Neural networks; prior knowledge; Semantics; Skeleton; skeleton action recognition; temporal modeling; prior knowledge; skeleton action recognition; temporal modeling
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s23249738

In action recognition, obtaining skeleton data from human poses is valuable. This process can help eliminate negative effects of environmental noise, including changes in background and lighting conditions. Although GCN can learn unique action features, it fails to fully utilize the prior knowledge of human body structure and the coordination relations between limbs. To address these issues, this paper proposes a Multi-level Topological Channel Attention Network algorithm: Firstly, the Multi-level Topology and Channel Attention Module incorporates prior knowledge of human body structure using a coarse-to-fine approach, effectively extracting action features. Secondly, the Coordination Module utilizes contralateral and ipsilateral coordinated movements in human kinematics. Lastly, the Multi-scale Global Spatio-temporal Attention Module captures spatiotemporal features of different granularities and incorporates a causal convolution block and masked temporal attention to prevent non-causal relationships. This method achieved accuracy rates of 91.9% (Xsub), 96.3% (Xview), 88.5% (Xsub), and 90.3% (Xset) on NTU-RGB+D 60 and NTU-RGB+D 120, respectively.