RadarSNN: A Resource Efficient Gesture Sensing System Based on mm-Wave Radar

• Published in: IEEE transactions on microwave theory and techniques Vol. 70; no. 4; pp. 2451 - 2461
• Main Authors: Arsalan, Muhammad, Santra, Avik, Issakov, Vadim
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.04.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Chirp; Continuous radiation; Fast Fourier transformations; Fourier transforms; Gesture recognition; Gesture sensing; human–computer interface; Machine learning; Millimeter waves; Nengo; NengoDL; Neural networks; neuromorphic hardware; Neurons; Object detection; Pipelines; Radar; Sensors; Spectrogram; Spectrograms; spiking neural networks (SNNs)
• ISSN: 0018-9480; 1557-9670
• DOI: 10.1109/TMTT.2022.3148403

Radar offers a promising modality for enabling gesture recognition, which is a simple and intuitive alternative to click and touch-based human-computer interface. In this article, we propose a spiking neural network (SNN)-based hand gesture recognition with frequency-modulated continuous-wave 60-GHz radar. As preprocessing, the 2-D fast Fourier transform (FFT) is performed across fast time and slow time to generate a video of range-Doppler maps, which are then processed to generate range spectrograms, Doppler spectrograms, and angle spectrograms. The spike trains are fed into an SNN to classify the gesture that has been performed. We demonstrate that with few neurons, SNNs can achieve recognition accuracies close to 99.50% comparable to their deep learning counterparts for eight dynamic gestures. Moreover, the proposed model size is 75 kB, which is substantially smaller compared to the state-of-the-art models making it memory efficient. We also demonstrate using tSNE plots that SNNs can operate with lower embedding dimensions, implying that we can realize SNN with a small compute and memory footprint.