Gesture Recognition Using Wearable Sensors With Bi-Long Short-Term Memory Convolutional Neural Networks

• Published in: IEEE sensors journal Vol. 21; no. 13; pp. 15065 - 15079
• Main Authors: Nguyen-Trong, Khanh, Vu, Hoai Nam, Trung, Ngon Nguyen, Pham, Cuong
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.07.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: 1D convolutional neural network; Accelerometers; Artificial neural networks; bidirectional LSTM; Data mining; Datasets; Deep learning; Empirical analysis; Feature extraction; GesHome dataset; Gesture recognition; hand gesture recognition; home appliance control; Home appliances; Household appliances; human > machine interface; Intelligent sensors; Neural networks; Sensors; Short term; Smartwatches
• ISSN: 1530-437X; 1558-1748
• DOI: 10.1109/JSEN.2021.3074642

In this study, we propose a gesture recognition system that is applicable for controlling home appliances. We utilized sensors embedded inside common smart watches, such as accelerometers and gyroscopes, for alleviating the obtrusiveness to users. One-dimensional convolutional neural networks and bi-long short-term memory (1D-CNN-biLSTM) are proposed for analyzing, learning, and representing features from the sensor signals. In addition, a dataset of 18,000 gestures with 18 labels was collected from 20 subjects to verify our proposed methods. Notably, the proposed hand gesture vocabulary was found to be easy to learn for users. Moreover, it provides them with improved control over their home appliances. The results of an empirical experiment conducted on three public datasets in addition to our self-collected dataset (GesHome) indicate that the proposed 1D-CNN-biLSTM model can achieve an F1-score of 90% and outperforms previous state-of-the art methods. Moreover, a demonstrative system was employed to illustrate the efficiency of the proposed model for home appliance control in a real-world scenario.