Stable Responsive EMG Sequence Prediction and Adaptive Reinforcement With Temporal Convolutional Networks

• Published in: IEEE transactions on biomedical engineering Vol. 67; no. 6; pp. 1707 - 1717
• Main Authors: Betthauser, Joseph L., Krall, John T., Bannowsky, Shain G., Levay, Gyorgy, Kaliki, Rahul R., Fifer, Matthew S., Thakor, Nitish V.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.06.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Amputation; amputee; Amputees; Artificial Limbs; Biomedical monitoring; Computational modeling; Control systems; Convolution; Decoding; ED-TCN; Electromyographic (EMG); Electromyography; Hand; Humans; latency; Movement; Networks; Neural networks; Pattern recognition; Performance prediction; Prediction models; Predictive models; Prostheses; Prosthetics; Recurrent neural networks; reinforcement; Reinforcement learning; Reproducibility of Results; sequence; Signal classification; stability; Stability analysis; temporal convolutional network (TCN); Training; Wrist
• ISSN: 0018-9294; 1558-2531; 1558-2531
• DOI: 10.1109/TBME.2019.2943309

Prediction of movement intentions from electromyographic (EMG) signals is typically performed with a pattern recognition approach, wherein a short dataframe of raw EMG is compressed into an instantaneous feature-encoding that is meaningful for classification. However, EMG signals are time-varying, implying that a frame-wise approach may not sufficiently incorporate temporal context into predictions, leading to erratic and unstable prediction behavior. Objective: We demonstrate that sequential prediction models and, specifically, temporal convolutional networks are able to leverage useful temporal information from EMG to achieve superior predictive performance. Methods: We compare this approach to other sequential and frame-wise models predicting 3 simultaneous hand and wrist degrees-of-freedom from 2 amputee and 13 non-amputee human subjects in a minimally constrained experiment. We also compare these models on the publicly available Ninapro and CapgMyo amputee and non-amputee datasets. Results: Temporal convolutional networks yield predictions that are more accurate and stable <inline-formula><tex-math notation="LaTeX">(p < 0.001)</tex-math></inline-formula> than frame-wise models, especially during inter-class transitions, with an average response delay of 4.6 ms <inline-formula><tex-math notation="LaTeX">(p < 0.001)</tex-math></inline-formula> and simpler feature-encoding. Their performance can be further improved with adaptive reinforcement training. Significance: Sequential models that incorporate temporal information from EMG achieve superior movement prediction performance and these models allow for novel types of interactive training. Conclusions: Addressing EMG decoding as a sequential modeling problem will lead to enhancements in the reliability, responsiveness, and movement complexity available from prosthesis control systems.