A CNN-LSTM Hybrid Model for Wrist Kinematics Estimation Using Surface Electromyography

• Published in: IEEE transactions on instrumentation and measurement Vol. 70; pp. 1 - 9
• Main Authors: Bao, Tianzhe, Zaidi, Syed Ali Raza, Xie, Shengquan, Yang, Pengfei, Zhang, Zhi-Qiang
• Format: Journal Article
• Language: English
• Published: New York IEEE 2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Computer architecture; Convolutional neural network (CNN); deep learning (DL); Electromyography; Estimation; Feature extraction; hybrid model; Kinematics; Logic gates; long short-term memory (LSTM) network; Machine learning; Muscles; Muscular function; Myoelectric control; Myoelectricity; Neural networks; Nonlinear dynamics; Regression analysis; Spatial data; surface electromyography (sEMG); Training; Wrist; wrist kinematics estimation
• ISSN: 0018-9456; 1557-9662
• DOI: 10.1109/TIM.2020.3036654

Convolutional neural network (CNN) has been widely exploited for simultaneous and proportional myoelectric control due to its capability of deriving informative, representative, and transferable features from surface electromyography (sEMG). However, muscle contractions have strong temporal dependencies, but conventional CNN can only exploit spatial correlations. Considering that the long short-term memory (LSTM) neural network is able to capture long-term and nonlinear dynamics of time-series data, in this article, we propose a CNN-LSTM hybrid model to fully explore the temporal-spatial information in sEMG. First, CNN is utilized to extract deep features from the sEMG spectrum, and then, these features are processed via LSTM-based sequence regression to estimate wrist kinematics. Six healthy participants are recruited for the participatory collection and motion analysis under various experimental setups. Estimation results in both intrasession and intersession evaluations illustrate that CNN-LSTM significantly outperforms CNN, LSTM, and several representative machine learning approaches, particularly when complex wrist movements are activated.