Research on decoupled adaptive graph convolution networks based on skeleton data for action recognition

• Published in: Pattern analysis and applications : PAA Vol. 27; no. 4
• Main Authors: Deng, Haigang, Lin, Guocheng, Li, Chengwei, Wang, Chuanxu
• Format: Journal Article
• Language: English
• Published: London Springer London 01.12.2024; Springer Nature B.V
• Subjects: Adaptive algorithms; Artificial neural networks; Classification; Computer Science; Convolution; Feature extraction; Feature recognition; Graphical representations; Original Article; Pattern Recognition; Spatiotemporal data; Skeleton based action recognition; Residual multi-scale temporal convolution network; Decoupled adaptive graph convolutional network; Decoupled head of classification lay
• ISSN: 1433-7541; 1433-755X
• DOI: 10.1007/s10044-024-01319-3

Graph convolutional network is apt for feature extraction in terms of non-Euclidian human skeleton data, but its adjacency matrix is fixed and the receptive field is small, which results in bias representation for skeleton intrinsic information. In addition, the operation of mean pooling on spatio-temporal features in classification layer will result in losing information and degrade recognition accuracy. To this end, the Decoupled Adaptive Graph Convolutional Network (DAGCN) is proposed. Specifically, a multi-level adaptive adjacency matrix is designed, which can dynamically obtain the rich correlation information among the skeleton nodes by a non-local adaptive algorithm. Whereafter, a new Residual Multi-scale Temporal Convolution Network (RMTCN) is proposed to fully extract temporal feature of the above decoupled skeleton dada. For the second problem in classification, we decompose the spatio-temporal features into three parts as spatial, temporal, spatio-temporal information, they are averagely pooled respectively, and added together for classification, denoted as STMP (spatio-temporal mean pooling) module. Experimental results show that our algorithm achieves accuracy of 96.5%, 90.6%, 96.4% on NTU-RGB+D60, NTU-RGB+D120 and NW-UCLA data sets respectively.