Wrist-Worn Hand Gesture Recognition While Walking via Transfer Learning

• Published in: IEEE journal of biomedical and health informatics Vol. 26; no. 3; pp. 952 - 961
• Main Authors: Kang, Peiqi, Li, Jinxuan, Fan, Bingfei, Jiang, Shuo, Shull, Peter B.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.03.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Algorithms; Bioinformatics; body movement; Decomposition; Error detection; Gesture recognition; Gestures; Hand; Hand gesture recognition; Heuristic algorithms; Humans; IMU; Learning; Legged locomotion; Machine Learning; Sensors; signal decomposition; Signal resolution; Transfer learning; Walking; wearable; Wrist
• ISSN: 2168-2194; 2168-2208; 2168-2208
• DOI: 10.1109/JBHI.2021.3100099

Walking, one of the most common daily activities, causes unwanted movement artifacts which can significantly deteriorate hand gesture recognition accuracy. However, traditional hand gesture recognition algorithms are typically developed and validated with wrist-worn devices only during static human poses, neglecting the critical importance of dynamic effects on gesture accuracy. Thus, we developed and validated a signal decomposition approach via empirical mode decomposition to accurately segment target gestures from coupled raw signals during dynamic walking and a transfer learning method based on distribution adaptation to enable gesture recognition through domain transfer between dynamic walking and static standing scenarios. Ten healthy subjects performed seven hand gestures during both walking and standing experiments while wearing an IMU wrist-worn device. Experimental results showed that the signal decomposition approach reduced the gesture detection error by 83.8%, and the transfer learning approach (20% transfer rate) improved hand gesture recognition accuracy by 15.1%. This ground-breaking work demonstrates the feasibility of hand gesture recognition while walking via wrist-worn sensing. These findings serve to inform real-life and ubiquitous adoption of wrist-worn hand gesture recognition for intuitive human-machine interaction in dynamic walking situations.