A Novel Network Architecture and Training Strategies for Camera-Radar 3D Detection

Intelligent vehicles rely on millimeter-wave radar and machine vision to perceive their surroundings. However, the considerable differences in the features of radar point clouds and those of image pixels make it difficult for models to perform effective fusion. Moreover, high-frequency noise in imag...

Full description

Saved in:
Bibliographic Details
Published in2023 International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan) pp. 411 - 412
Main Authors Jhong, Sin-Ye, Lin, Hsin-Chun, Weng, Xu-Xiang, Xie, Ting-Feng, Lin, Han-Wei, Chen, Yung-Yao
Format Conference Proceeding
LanguageEnglish
Published IEEE 17.07.2023
Subjects
3D object detection
camera-radar fusion
Feature extraction
intelligent vehicle sensing
Millimeter wave radar
multi-sensor fusion
Object detection
Radar detection
Radar imaging
Three-dimensional displays
Training
Online AccessGet full text

Cover

More Information
Summary:Intelligent vehicles rely on millimeter-wave radar and machine vision to perceive their surroundings. However, the considerable differences in the features of radar point clouds and those of image pixels make it difficult for models to perform effective fusion. Moreover, high-frequency noise in images can impede the extraction of meaningful features. This paper proposes a novel 3D object detection method that combines millimeter-wave radar and RGB camera data. Our approach includes a gaussian filter for preprocessing, a hierarchical model architecture for fusing radar and image information, and a training stabilization strategy. We evaluated our method using the challenging NuScenes and Taiwan street databases and found that it outperformed the popular CenterFusion model in terms of detection performance. In addition, our method is applicable to a variety of scenarios in Taiwan.
ISSN:2575-8284
DOI:10.1109/ICCE-Taiwan58799.2023.10226927