A Combination Model of Shifting Joint Angle Changes with 3D-Deep Convolutional Neural Network to Recognize Human Activity

• Published in: IEEE transactions on neural systems and rehabilitation engineering Vol. 32; p. 1
• Main Authors: Rahayu, Endang Sri, Yuniarno, Eko Mulyanto, Ketut Eddy Purnama, I, Purnomo, Mauridhi Hery
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Activity patterns; Artificial neural networks; Classification; Combination model; Convolutional neural networks; Data models; Datasets; deep convolutional neural network; Deep Learning; Hidden Markov models; Human Activities; Human activity recognition; Human motion; Humans; Joints; Motion; Movement; Neural networks; Neural Networks, Computer; Performance evaluation; Postal services; shifting joint angles; Spatiotemporal phenomena; Three dimensional models; Three-dimensional displays
• ISSN: 1534-4320; 1558-0210
• DOI: 10.1109/TNSRE.2024.3371474

Research in the field of human activity recognition is very interesting due to its potential for various applications such as in the field of medical rehabilitation. The need to advance its development has become increasingly necessary to enable efficient detection and response to a wide range of movements. Current recognition methods rely on calculating changes in joint distance to classify activity patterns. Therefore, a different approach is required to identify the direction of movement to distinguish activities exhibiting similar joint distance changes but differing motion directions, such as sitting and standing. The research conducted in this study focused on determining the direction of movement using an innovative joint angle shift approach. By analyzing the joint angle shift value between specific joints and reference points in the sequence of activity frames, the research enabled the detection of variations in activity direction. The joint angle shift method was combined with a Deep Convolutional Neural Network (DCNN) model to classify 3D datasets encompassing spatial-temporal information from RGB-D video image data. Model performance was evaluated using the confusion matrix. The results show that the model successfully classified nine activities in the Florence 3D Actions dataset, including sitting and standing, obtaining an accuracy of (96.72 ± 0.83)%. In addition, to evaluate its robustness, this model was tested on the UTKinect Action3D dataset, obtaining an accuracy of 97.44%, proving that state-of-the-art performance has been achieved.