Data glove-based gesture recognition using CNN-BiLSTM model with attention mechanism

• Published in: PloS one Vol. 18; no. 11; p. e0294174
• Main Authors: Wu, Jiawei, Ren, Peng, Song, Boming, Zhang, Ran, Zhao, Chen, Zhang, Xiao
• Format: Journal Article
• Language: English
• Published: San Francisco Public Library of Science 17.11.2023; Public Library of Science (PLoS)
• Subjects: Accuracy; Algorithms; Artificial intelligence; Artificial neural networks; Automation; Cameras; Classification; Computer applications; Data collection; Data entry; Deep learning; Evaluation; Experiments; Forecasts and trends; Gesture recognition; Gloves; Long short-term memory; Medical research; Neural networks; Occlusion; Semantics; Sensors; Sign language; Temporal variations; User needs; Virtual reality; Wavelet transforms; Wearable computers; China
• ISSN: 1932-6203; 1932-6203
• DOI: 10.1371/journal.pone.0294174

As a novel form of human machine interaction (HMI), hand gesture recognition (HGR) has garnered extensive attention and research. The majority of HGR studies are based on visual systems, inevitably encountering challenges such as depth and occlusion. On the contrary, data gloves can facilitate data collection with minimal interference in complex environments, thus becoming a research focus in fields such as medical simulation and virtual reality. To explore the application of data gloves in dynamic gesture recognition, this paper proposes a data glove-based dynamic gesture recognition model called the Attention-based CNN-BiLSTM Network (A-CBLN). In A-CBLN, the convolutional neural network (CNN) is employed to capture local features, while the bidirectional long short-term memory (BiLSTM) is used to extract contextual temporal features of gesture data. By utilizing attention mechanisms to allocate weights to gesture features, the model enhances its understanding of different gesture meanings, thereby improving recognition accuracy. We selected seven dynamic gestures as research targets and recruited 32 subjects for participation. Experimental results demonstrate that A-CBLN effectively addresses the challenge of dynamic gesture recognition, outperforming existing models and achieving optimal gesture recognition performance, with the accuracy of 95.05% and precision of 95.43% on the test dataset.