A long short-term recurrent spatial-temporal fusion for myoelectric pattern recognition

• Published in: Expert systems with applications Vol. 178; p. 114977
• Main Authors: Khushaba, Rami N., Scheme, Erik, Al-Timemy, Ali H., Phinyomark, Angkoon, Taee, Ahmed Al, Al-Jumaily, Adel
• Format: Journal Article
• Language: English
• Published: New York Elsevier Ltd 15.09.2021; Elsevier BV
• Subjects: Algorithms; Artificial neural networks; Computing costs; Correlation; Deep learning; Electromyogram; Feature extraction; LSTM; Machine learning; Myoelectricity; Nonlinear dynamics; Pattern recognition; Prostheses; Spatial filtering; Temporal-spatial correlations; Electromyogram; Feature extraction; LSTM; Pattern recognition; Temporal-spatial correlations
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2021.114977

•Deep learning concepts borrowed for building a powerful feature extraction framework.•A novel recurrent spatial-temporal fusion demonstrates a significant performance.•A bi-directional framework is also proposed equipped with simple TD features.•Results are benchmarked against several algorithms across 82 subjects.•RSTF significantly outperforms all other methods, including LSTM and BiLSTM. Current state-of-the-art myoelectric interfaces employ traditional pattern recognition (PR) algorithms to decode the Electromyogram (EMG) signals into hand movements for controlling artificial limbs. Recently, deep learning (DL) models have also been exploited for EMG feature learning/extraction. Models like Convolutional Neural Networks (CNN), which capture the spatial correlations, and Long Short-Term Memory (LSTM), which capture the non-linear temporal dynamics of EMG time-series data, have been shown to outperform traditional EMG PR systems. Nevertheless, the large number of model parameters, long training times, and large amounts of data required to train these DL models remain limiting factors that may hinder their translation into clinically viable prostheses. Consequently, rather than applying DL directly, this paper leverages concepts derived from these models to build upon our proposed concept of a Fusion of Time Domain Descriptors (FTDD). The FTDD are augmented with Range Spatial Filtering (RSF) to capture the spatial correlations and combined into an LSTM-style framework. This process, denoted as Recurrent Spatial-Temporal Fusion (RSTF), can be applied in combination with any traditional feature extraction method to exploit temporal and spatial correlations, with the potential for bi-directional applications. The advantages of the proposed RSTF method include (1) the memory concept, capturing long- and short-term spatial and temporal dependencies of the EMG signals, (2) significantly improved performance outperforming other state-of-the-art models and (3) the simplicity and the fairly low computational costs for feature extraction. Results are bench-marked against several feature extraction methods, proving the power of the RSTF using data from 82 subjects from five EMG databases with varying recording characteristics. The proposed method significantly outperforms all other methods tested for EMG pattern recognition, including a deep LSTM and other CNN methods previously reported in the literature and at a fracture of the computational cost. On the most challenging dataset, improvements of as much as 15% were found.