Recognition of Hand Gestures Based on EMG Signals with Deep and Double-Deep Q-Networks

• Published in: Sensors (Basel, Switzerland) Vol. 23; no. 8; p. 3905
• Main Authors: Valdivieso Caraguay, Ángel Leonardo, Vásconez, Juan Pablo, Barona López, Lorena Isabel, Benalcázar, Marco E
• Format: Journal Article
• Language: English
• Published: Switzerland MDPI AG 12.04.2023; MDPI
• Subjects: Accuracy; Algorithms; Analysis; Cameras; Classification; Computational linguistics; Datasets; Deep Q-Network; Double-Deep Q-Network; Elbow; Electromyography; Electromyography - methods; Gesture recognition; Gestures; Hand; hand gesture recognition; Humans; Language processing; Machine learning; Memory, Long-Term; Methods; Muscle function; Natural language interfaces; Neural networks; Neural Networks, Computer; Online education; Prostheses; Reagents; reinforcement learning; Sensors; Supervised learning; New York; Double-Deep Q-Network; Deep Q-Network; electromyography; hand gesture recognition; reinforcement learning
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s23083905

In recent years, hand gesture recognition (HGR) technologies that use electromyography (EMG) signals have been of considerable interest in developing human-machine interfaces. Most state-of-the-art HGR approaches are based mainly on supervised machine learning (ML). However, the use of reinforcement learning (RL) techniques to classify EMGs is still a new and open research topic. Methods based on RL have some advantages such as promising classification performance and online learning from the user's experience. In this work, we propose a user-specific HGR system based on an RL-based agent that learns to characterize EMG signals from five different hand gestures using Deep Q-network (DQN) and Double-Deep Q-Network (Double-DQN) algorithms. Both methods use a feed-forward artificial neural network (ANN) for the representation of the agent policy. We also performed additional tests by adding a long-short-term memory (LSTM) layer to the ANN to analyze and compare its performance. We performed experiments using training, validation, and test sets from our public dataset, EMG-EPN-612. The final accuracy results demonstrate that the best model was DQN without LSTM, obtaining classification and recognition accuracies of up to 90.37%±10.7% and 82.52%±10.9%, respectively. The results obtained in this work demonstrate that RL methods such as DQN and Double-DQN can obtain promising results for classification and recognition problems based on EMG signals.