ViT-MDHGR: Cross-Day Reliability and Agility in Dynamic Hand Gesture Prediction via HD-sEMG Signal Decoding

• Published in: IEEE journal of selected topics in signal processing Vol. 18; no. 3; pp. 419 - 430
• Main Authors: Hu, Qin, Azar, Golara Ahmadi, Fletcher, Alyson, Rangan, Sundeep, Atashzar, S. Farokh
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.04.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Adaptation models; Biomedical signal processing; Calibration; cross-day HGR; Data models; Decoding; Electrodes; Electromyography; Feasibility studies; Feature extraction; Gesture recognition; hand gesture recognition (HGR); Human-computer interface; Human-robot interaction; Human-robot interactions; minimal calibration; Myoelectric control; Myoelectricity; Performance degradation; Prostheses; Real time; Solid modeling; surface electromyography (sEMG); Testing; Training; vision transformer; Vision transformers
• DOI: 10.1109/JSTSP.2024.3402340
• ISSN: 1932-4553

Surface electromyography (sEMG) and high-density sEMG (HD-sEMG) biosignals have been extensively investigated for myoelectric control of prosthetic devices, neurorobotics, and more recently human-computer interfaces because of their capability for hand gesture recognition/prediction in a wearable and non-invasive manner. High intraday (same-day) performance has been reported. However, the interday performance (separating training and testing days) is substantially degraded due to the poor generalizability of conventional approaches over time, hindering the application of such techniques in real-life practices. There are limited recent studies on the feasibility of multi-day hand gesture recognition. The existing studies face a major challenge: the need for long sEMG epochs makes the corresponding neural interfaces impractical due to the induced delay in myoelectric control. This paper proposes a compact ViT-based network for multi-day dynamic hand gesture prediction. We tackle the main challenge as the proposed model only relies on very short HD-sEMG signal windows (i.e., 50 ms, accounting for only one-sixth of the convention for real-time myoelectric implementation), boosting agility and responsiveness. Our proposed model can predict 11 dynamic gestures for 20 subjects with an average accuracy of over 71% on the testing day, 3-25 days after training. Moreover, when calibrated on just a small portion of data from the testing day, the proposed model can achieve over 92% accuracy by retraining less than 10% of the parameters for computational efficiency.