Pairwise Adjacency Matrix on Spatial Temporal Graph Convolution Network for Skeleton-Based Two-Person Interaction Recognition

• Published in: Proceedings - International Conference on Image Processing pp. 2166 - 2170
• Main Authors: Yang, Chao-Lung, Setyoko, Aji, Tampubolon, Hendrik, Hua, Kai-Lung
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.10.2020
• Subjects: Indexes; Large scale integration; pairwise adjacency matrix; skeleton-based action recognition; spatialtemporal graph convolution network; two-person-interaction
• ISSN: 2381-8549
• DOI: 10.1109/ICIP40778.2020.9190680

Spatial-temporal graph convolutional networks (ST-GCN) have achieved outstanding performances on human action recognition, however, it might be less superior on a two-person interaction recognition (TPIR) task due to the relationship of each skeleton is not considered. In this study, we present an improvement of the STGCN model that focused on TPIR by employing the pairwise adjacency matrix to capture the relationship of person-person skeletons (ST-GCN-PAM). To validate the effectiveness of the proposed ST-GCN-PAM model on TPIR, experiments were conducted on NTU RGB+D120. Additionally, the model was also examined on the Kinetics dataset and NTU RGB+D60. The results show that the proposed ST-GCN-PAM outperforms the-state-of-the-art methods on mutual action of NTU RGB+D120 by achieving 83.28% (cross-subject) and 88.31% (cross-view) accuracy. The model is also superior to the original ST-GCN on the multi-human action of the Kinetics dataset by achieving 41.68% in Top-l and 88.91% in Top-5.