A Fully Embedded Adaptive Real-Time Hand Gesture Classifier Leveraging HD-sEMG and Deep Learning

• Published in: IEEE transactions on biomedical circuits and systems Vol. 14; no. 2; pp. 232 - 243
• Main Authors: Tam, Simon, Boukadoum, Mounir, Campeau-Lecours, Alexandre, Gosselin, Benoit
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.04.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Armband; Artificial intelligence; Artificial Limbs; Artificial neural networks; Calibration; Circuit boards; Co-design; Control; Convolution; Data processing; Data transmission; Deep Learning; Edge Computing; Electrodes; Electromyography; Electromyography - instrumentation; Equipment Design; Forearm; Forearm - physiology; Gesture; Gesture recognition; Gestures; Hand - physiology; HD-EMG; Homogeneity; Humans; Latency; Machine Learning; Motion; Muscle contraction; Muscle, Skeletal - physiology; Muscles; Muscular function; Myoelectric; Neural Network; Neural networks; Printed circuits; Prostheses; Prosthetic Hand; Prosthetics; Real time operation; Real-Time; Real-time systems; Recognition; Signal processing; Signal Processing, Computer-Assisted - instrumentation
• ISSN: 1932-4545; 1940-9990
• DOI: 10.1109/TBCAS.2019.2955641

This paper presents a real-time fine gesture recognition system for multi-articulating hand prosthesis control, using an embedded convolutional neural network (CNN) to classify hand-muscle contractions sensed at the forearm. The sensor consists in a custom non-intrusive, compact, and easy-to-install 32-channel high-density surface electromyography (HDsEMG) electrode array, built on a flexible printed circuit board (PCB) to allow wrapping around the forearm. The sensor provides a low-noise digitization interface with wireless data transmission through an industrial, scientific and medical (ISM) radio link. An original frequency-time-space cross-domain preprocessing method is proposed to enhance gesture-specific data homogeneity and generate reliable muscle activation maps, leading to 98.15% accuracy when using a majority vote over 5 subsequent inferences by the proposed CNN. The obtained real-time gesture recognition, within 100 to 200 ms, and CNN properties show reliable and promising results to improve on the state-of-the-art of commercial hand prostheses. Moreover, edge computing using a specialized embedded artificial intelligence (AI) platform ensures reliable, secure and low latency real-time operation as well as quick and easy access to training, fine-tuning and calibration of the neural network. Co-design of the signal processing, AI algorithms and sensing hardware ensures a reliable and power-efficient embedded gesture recognition system.