Cyber Mobility Mirror: A Deep Learning-Based Real-World Object Perception Platform Using Roadside LiDAR

• Published in: IEEE transactions on intelligent transportation systems Vol. 24; no. 9; pp. 9476 - 9489
• Main Authors: Bai, Zhengwei, Nayak, Saswat P., Zhao, Xuanpeng, Wu, Guoyuan, Barth, Matthew J., Qi, Xuewei, Liu, Yongkang, Sisbot, Emrah Akin, Oguchi, Kentaro
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.09.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: 3D object detection; Automation; Cloud computing; cooperative driving automation; deep learning; Driving conditions; Field operational system; Field tests; Laser radar; Localization; Microprocessors; multi-object tracking; Object detection; Object recognition; Occlusion; Perception; Point cloud compression; Real time; Recall; Roadsides; Root-mean-square errors; Sensors; Three-dimensional displays; Tracking; Traffic; Traffic control; Transportation systems
• ISSN: 1524-9050; 1558-0016
• DOI: 10.1109/TITS.2023.3268281

Object perception plays a fundamental role in Cooperative Driving Automation (CDA) which is regarded as a revolutionary promoter for next-generation transportation systems. However, the vehicle-based perception may suffer from the limited sensing range and occlusion as well as low penetration rates in connectivity. In this paper, we propose Cyber Mobility Mirror (CMM), a next-generation real-world object perception system for 3D object detection, tracking, localization, and reconstruction, to explore the potential of roadside sensors for enabling CDA in the real world. The CMM system consists of six main components: i) the data pre-processor to retrieve and preprocess the raw data; ii) the roadside 3D object detector to generate 3D detection results; iii) the multi-object tracker to identify detected objects; iv) the global locator to generate geo-localization information; v) the mobile-edge-cloud-based communicator to transmit perception information to equipped vehicles, and vi) the onboard advisor to reconstruct and display the real-time traffic conditions. An automatic perception evaluation approach is proposed to support the assessment of data-driven models without human-labeling requirements and a CMM field-operational system is deployed at a real-world intersection to assess the performance of the CMM. Results from field tests demonstrate that our CMM prototype system can achieve 96.99% precision and 83.62% recall for detection and 73.55% ID-recall for tracking. High-fidelity real-time traffic conditions (at the object level) can be geo-localized with a root-mean-square error (RMSE) of <inline-formula> <tex-math notation="LaTeX">0.69m </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">0.33m </tex-math></inline-formula> for lateral and longitudinal direction, respectively, and displayed on the GUI of the equipped vehicle with a frequency of <inline-formula> <tex-math notation="LaTeX">3-4 Hz </tex-math></inline-formula>.