Pseudo-Mono for Monocular 3D Object Detection in Autonomous Driving

• Published in: IEEE transactions on circuits and systems for video technology Vol. 33; no. 8; pp. 3962 - 3975
• Main Authors: Tao, Chongben, Cao, Jiecheng, Wang, Chen, Zhang, Zufeng, Gao, Zhen
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.08.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: 3D object detection; Ablation; Algorithms; Decoding; Feature extraction; Feature maps; Heuristic algorithms; Image enhancement; Image processing; multi-scale feature; Object detection; Object recognition; sparse anchor point; stereo images; Three-dimensional displays; transformer; Transformers; Visualization
• DOI: 10.1109/TCSVT.2023.3237579
• ISSN: 1051-8215

Current monocular 3D object detection algorithms generally suffer from inaccurate depth estimation, which leads to reduction of detection accuracy. The depth error from image-to-image generation for the stereo view is insignificant compared with the gap in single-image generation. Therefore, a novel pseudo-monocular 3D object detection framework is proposed, which is called Pseudo-Mono. Particularly, stereo images are brought into monocular 3D detection. Firstly, stereo images are taken as input, then a lightweight depth predictor is used to generate the depth map of input images. Secondly, the left input images obtained from stereo camera are used as subjects, which generate enhanced visual feature and multi-scale depth feature by depth indexing and feature matching probabilities, respectively. Finally, sparse anchors set by the foreground probability maps and the multi-scale feature maps are used as reference points to find the suitable initialization approach of object query. The encoded visual feature is adopted to enhance object query for enabling deep interaction between visual feature and depth feature. Compared with popular monocular 3D object detection methods, Pseudo-Mono is able to achieve richer fine-grained information without additional data input. Extensive experimental results on the datasets of KITTI, NuScenes, and MS-COCO demonstrate the generalizability and portability of the proposed method. The effectiveness and efficiency of Pseudo-Mono have been demonstrated by extensive ablation experiments. Experiments on a real vehicle platform have shown that the proposed method maintains high performance in complex real-world environments.