Human Action Recognition Based on The Skeletal Pairwise Dissimilarity

• Published in: Kompʹûternaâ optika Vol. 49; no. 3; pp. 493 - 503
• Main Authors: E.E. Surkov, O.S. Seredin, A.V. Kopylov
• Format: Journal Article
• Language: English
• Published: Samara National Research University 01.06.2025
• Subjects: activity map; basic assembly; cnn; human action recognition; inner-channel attention; pairwise dissimilarity measure
• DOI: 10.18287/2412-6179-CO-1522
• ISSN: 0134-2452

The main idea of the paper is to apply the principles of featureless pattern recognition to human activity recognition problem. The article presents the human figure representing approach based on pairwise dissimilarity function of skeletal models and a set of reference objects, also known as a basic assembly. The paper includes a basic assembly analysis and we propose the method for selecting the least-correlated basic objects. The video sequence proposed for analysis of human activity within frames is represented as an activity map. The activity map is a result of computing the pairwise dissimilarity function between skeletal models from the video sequence and the basic assembly of skeletons. The paper conducts frame-by-frame annotation of activities in the TST Fall Detection v2 database, such as standing, sitting, lying, walking, falling, post-fall lying, grasp, ungrasp. A convolutional neural network based on the ResNetV2 with the SE-block is proposed to solve the activity recognition problem. SE-block allows to detect inter-channel dependencies and selecting the most important features. Additionally, we prepare a data for training, determine an optimal hyperparameters of the neural network model. Experimental results of human activity recognition on the TST Fall Detection v2 database using the Leave-one-person-out procedure are provided. Furthermore, the paper presents a frame-by-frame assessment of the quality of human activity recognition, achieving an accuracy exceeding 83%.