Meta-Transfer-Learning-Based Multimodal Human Pose Estimation for Lower Limbs

• Published in: Sensors (Basel, Switzerland) Vol. 25; no. 5; p. 1613
• Main Authors: Du, Guoming, Zhu, Haiqi, Ding, Zhen, Huang, Hong, Bie, Xiaofeng, Jiang, Feng
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 06.03.2025; MDPI
• Subjects: Accuracy; Algorithms; Computational linguistics; Deep learning; Electromyography - methods; Exoskeleton Device; human pose estimation; Humans; knowledge fusion; Knowledge representation; Language processing; Lower Extremity - physiology; Machine Learning; meta learning; Motion capture; multimodal; Natural language interfaces; Neural networks; Posture - physiology; Robotics; sEMG; Sensors; transfer learning; Walking; Wearable Electronic Devices; United States; China; New Jersey; Germany; United States > US; transfer learning; sEMG; multimodal; human pose estimation; knowledge fusion; meta learning
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s25051613

Accurate and reliable human pose estimation (HPE) is essential in interactive systems, particularly for applications requiring personalized adaptation, such as controlling cooperative robots and wearable exoskeletons, especially for healthcare monitoring equipment. However, continuously maintaining diverse datasets and frequently updating models for individual adaptation are both resource intensive and time-consuming. To address these challenges, we propose a meta-transfer learning framework that integrates multimodal inputs, including high-frequency surface electromyography (sEMG), visual-inertial odometry (VIO), and high-precision image data. This framework improves both accuracy and stability through a knowledge fusion strategy, resolving the data alignment issue, ensuring seamless integration of different modalities. To further enhance adaptability, we introduce a training and adaptation framework with few-shot learning, facilitating efficient updating of encoders and decoders for dynamic feature adjustment in real-time applications. Experimental results demonstrate that our framework provides accurate, high-frequency pose estimations, particularly for intra-subject adaptation. Our approach enables efficient adaptation to new individuals with only a few new samples, providing an effective solution for personalized motion analysis with minimal data.