sEMG-Based Gesture Recognition With Embedded Virtual Hand Poses and Adversarial Learning

• Published in: IEEE access Vol. 7; pp. 104108 - 104120
• Main Authors: Hu, Yu, Wong, Yongkang, Dai, Qingfeng, Kankanhalli, Mohan, Geng, Weidong, Li, Xiangdong
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Assistive technology; Density; Electrodes; Feature extraction; generative adversarial learning; Gesture recognition; Hand gesture recognition; Learning; Machine learning; myoelectric control; surface electromyography (sEMG); User experience; virtual hand pose
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2019.2930005

To improve the accuracy of surface electromyography (sEMG)-based gesture recognition, we present a novel hybrid approach that combines real sEMG signals with corresponding virtual hand poses. The virtual hand poses are generated by means of a proposed cross-modal association model constructed based on the adversarial learning to capture the intrinsic relationship between the sEMG signals and the hand poses. We report comprehensive evaluations of the proposed approach for both frame- and window-based sEMG gesture recognitions on seven-sparse-multichannel and four-high-density-benchmark databases. The experimental results show that the proposed approach achieves significant improvements in sEMG-based gesture recognition compared to existing works. For frame-based sEMG gesture recognition, the recognition accuracy of the proposed framework is increased by an average of +5.2% on the sparse multichannel sEMG databases and by an average of +6.7% on the high-density sEMG databases compared to the existing methods. For window-based sEMG gesture recognition, the state-of-the-art recognition accuracies on three of the high-density sEMG databases are already higher than 99%, i.e., almost saturated; nevertheless, we achieve a +0.2% improvement. For the remaining eight sEMG databases, the average improvement with the proposed framework for the window-based approach is +2.5%.