Fine-Grained Assessment of Upper-Limb Bradykinesia Through Multimodal Feature Enhancement and Deep Learning

• Published in: IEEE transactions on human-machine systems Vol. 55; no. 4; pp. 508 - 518
• Main Authors: Lin, Fang, Wang, Zhelong, Li, Zhenglin, Zhao, Hongyu, Shi, Xin, Liu, Ruichen, Li, Jiaxi, Peng, Daoyong, Ru, Bo
• Format: Journal Article
• Language: English
• Published: IEEE 01.08.2025
• Subjects: Bradykinesia; data augmentation; Data mining; Diseases; Feature extraction; fine-grained classification; Hands; Long short term memory; Motors; multimodal feature; Sensors; Skeleton; Videos; Wrist
• ISSN: 2168-2291; 2168-2305
• DOI: 10.1109/THMS.2025.3570704

Bradykinesia is a hallmark symptom of Parkinson's disease (PD) that significantly affects patients' functional abilities and quality of life. This study proposed a fine-grained classification method for evaluating the level of bradykinesia in PD patients. Based on inertial signals, surface electromyographic (sEMG) signals, and videos obtained from 40 PD patients and 13 healthy subjects, the proposed data preprocessing method extracts 69-D features from inertial and sEMG (IE) signals, and 7-D skeleton features from videos. A two-stream network, including IE stream, skeleton stream, and decision fusion module, was developed using long short-term memory, full convolutional neural networks, and fully connected neural networks. In addition, the IE stream incorporated a feature shrinking module to process high-dimensional features to reduce redundant features. Furthermore, an LSTM-variational autoencoders method was proposed for data augmentation of categories with fewer samples. The proposed method achieved higher recognition rates (pro/supination movements of hands: 85.51%, finger tapping: 88.06%, hand movements: 90.00%) compared to other methods. With low-cost, compact and lightweight methods, bradykinesia in PD patients can be intelligently assessed, which will enhance patient management and treatment efficiency.