Gesture recognition system using reconstructed image from the acceleration sensor signal

• Published in: 2017 17th International Symposium on Communications and Information Technologies (ISCIT) pp. 1 - 6
• Main Authors: Yanzhe Zhao, Ting Jiang, Weixia Zou
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.09.2017
• Subjects: Acceleration; acceleration sensor; Feature extraction; Gesture recognition; handwritten gesture recognition; Hidden Markov models; Image reconstruction; Motion segmentation; reconstructed image; Softmax regression model; Wrist
• DOI: 10.1109/ISCIT.2017.8261169

This paper presents a new method for recognizing handwritten motions in 3-D space based on the reconstructed image from the signal of the three-axis acceleration (ACC) sensor of the smart mobile device. The proposed method is composed of gesture detection, data reconstruction and recognition. In our method, the start and end points of the meaningful gesture motions are detected automatically and the undefined hand motions are filtered out. The three-axis ACC data of the gesture segment is directly converted and reconstructed into gray-scale image. By reconstructing the ACC signal as an image, the detected gesture segments having different lengths are normalized and compressed. The classification and recognition of the gray-scale images representing different gestures are further carried out based on Softmax regression model. The recognition algorithm can recognize a large number of complex letter gestures with low computational cost. A library of 26 letter gestures with over 3000 samples is created to evaluate the proposed method, and the evaluation shows that the method achieves accuracy of 99.23% and the implemented recognition system on smart-phone shows that the average recognition time for a letter gesture is 15ms.