High Resolution Point Clouds from mmWave Radar

• Published in: arXiv.org
• Main Authors: Akarsh Prabhakara, Jin, Tao, Das, Arnav, Bhatt, Gantavya, Kumari, Lilly, Soltanaghaei, Elahe, Bilmes, Jeff, Kumar, Swarun, Rowe, Anthony
• Format: Paper Journal Article
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 16.07.2023
• Subjects: Computer Science - Networking and Internet Architecture; Computer Science - Robotics; High resolution; Image enhancement; Image processing; Lidar; Machine learning; Millimeter waves; Neural networks; Occlusion; Radar data; Radar imaging; Signal processing; Smoke; Spatial resolution; Training; Vision systems
• ISSN: 2331-8422
• DOI: 10.48550/arxiv.2206.09273

This paper explores a machine learning approach for generating high resolution point clouds from a single-chip mmWave radar. Unlike lidar and vision-based systems, mmWave radar can operate in harsh environments and see through occlusions like smoke, fog, and dust. Unfortunately, current mmWave processing techniques offer poor spatial resolution compared to lidar point clouds. This paper presents RadarHD, an end-to-end neural network that constructs lidar-like point clouds from low resolution radar input. Enhancing radar images is challenging due to the presence of specular and spurious reflections. Radar data also doesn't map well to traditional image processing techniques due to the signal's sinc-like spreading pattern. We overcome these challenges by training RadarHD on a large volume of raw I/Q radar data paired with lidar point clouds across diverse indoor settings. Our experiments show the ability to generate rich point clouds even in scenes unobserved during training and in the presence of heavy smoke occlusion. Further, RadarHD's point clouds are high-quality enough to work with existing lidar odometry and mapping workflows.