Dynamic Hand Gesture Recognition in In-Vehicle Environment Based on FMCW Radar and Transformer

• Published in: Sensors (Basel, Switzerland) Vol. 21; no. 19; p. 6368
• Main Authors: Zheng, Lianqing, Bai, Jie, Zhu, Xichan, Huang, Libo, Shan, Chewu, Wu, Qiong, Zhang, Lei
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 24.09.2021; MDPI
• Subjects: Accuracy; Algorithms; Artificial intelligence; Automobile safety; Classification; Continuous radiation; Datasets; deep learning; Doppler radar; Electromyography; Feature maps; FMCW radar; Gesture recognition; human-computer interaction; Neural networks; Principal components analysis; Sensors; Signal processing; Standardization; transformer; Wearable computers
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s21196368

Hand gesture recognition technology plays an important role in human-computer interaction and in-vehicle entertainment. Under in-vehicle conditions, it is a great challenge to design gesture recognition systems due to variable driving conditions, complex backgrounds, and diversified gestures. In this paper, we propose a gesture recognition system based on frequency-modulated continuous-wave (FMCW) radar and transformer for an in-vehicle environment. Firstly, the original range-Doppler maps (RDMs), range-azimuth maps (RAMs), and range-elevation maps (REMs) of the time sequence of each gesture are obtained by radar signal processing. Then we preprocess the obtained data frames by region of interest (ROI) extraction, vibration removal algorithm, background removal algorithm, and standardization. We propose a transformer-based radar gesture recognition network named RGTNet. It fully extracts and fuses the spatial-temporal information of radar feature maps to complete the classification of various gestures. The experimental results show that our method can better complete the eight gesture classification tasks in the in-vehicle environment. The recognition accuracy is 97.56%.