Cooperative Sensing and Wearable Computing for Sequential Hand Gesture Recognition

• Published in: IEEE sensors journal Vol. 19; no. 14; pp. 5775 - 5783
• Main Authors: Zhang, Xiaoliang, Yang, Ziqi, Chen, Taiyu, Chen, Diliang, Huang, Ming-Chun
• Format: Journal Article
• Language: English
• Published: New York IEEE 15.07.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Array signal processing; Computer memory; Computer vision; customizable pressure sensor arrays; Gesture recognition; Human computer interaction; Inertial platforms; LSTM; Machine learning; MYO armband; Optical measuring instruments; Pressure sensors; Recognition; Sensor arrays; Sequential hand gesture recognition; wearable sensor devices; Wearable technology
• ISSN: 1530-437X; 1558-1748
• DOI: 10.1109/JSEN.2019.2904595

Hand gestures recognition (HGR) has been considered as one of the crucial research fields of human-computer interaction (HCI). Computer vision is a very active research field in the HGR, traditional vision-based methods, which used camera and ultrasonic/optical sensor to collect the videos or images of the hand gestures shown by participants, have some limitations, such as fixed in-lab location, complex lighting conditions, and cluttered backgrounds. In order to provide new approaches, we described the development of a novel hand gesture recognition system that combined wearable armband and smart glove made by customizable pressure sensor arrays to detect sequential hand gestures. A deep learning technique long short-term memory (LSTM) algorithm had been computed to build an effective model to classify hand gestures by training and testing the collected inertial measurement unit (IMU), electromyographic (EMG), and finger and palm's pressure data. Furthermore, we built a relatively large database of ten sequential hand gestures consisted by five dynamic gestures and five air gestures collected from ten participants. Our experimental results showed an outstanding classification performance of the proposed LSTM algorithm. These findings have promising implications for sequential hand gesture recognition and the HCI research status.