Multiagent Multitraversal Multimodal Self-Driving: Open MARS Dataset

Large-scale datasets have fueled recent advancements in AI-based autonomous vehicle research. However, these datasets are usually collected from a single vehicle's one-time pass of a certain location, lacking multiagent inter-actions or repeated traversals of the same place. Such information co...

Full description

Saved in:
Bibliographic Details
Published in2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 22041 - 22051
Main Authors Li, Yiming, Li, Zhiheng, Chen, Nuo, Gong, Moonjun, Lyu, Zonglin, Wang, Zehong, Jiang, Peili, Feng, Chen
Format Conference Proceeding
LanguageEnglish
Published IEEE 16.06.2024
Subjects
Autonomous Driving
Collaboration
Computer vision
Data collection
Dataset and Benchmark
Laser radar
Mars
Neural Reconstruction
Semantics
Three-dimensional displays
Online AccessGet full text

Cover

More Information
Summary:Large-scale datasets have fueled recent advancements in AI-based autonomous vehicle research. However, these datasets are usually collected from a single vehicle's one-time pass of a certain location, lacking multiagent inter-actions or repeated traversals of the same place. Such information could lead to transformative enhancements in autonomous vehicles' perception, prediction, and planning capabilities. To bridge this gap, in collaboration with the self-driving company May Mobility, we present MARS dataset which unifies scenarios that enable MultiAgent, multitraveRSal, and multimodal autonomous vehicle re-search. More specifically, MARS is collected with a fleet of autonomous vehicles driving within a certain geographical area. Each vehicle has its own route and different vehicles may appear at nearby locations. Each vehicle is equipped with a LiDAR and surround-view RGB cameras. We curate two subsets in MARS: one facilitates collaborative driving with multiple vehicles simultaneously present at the same location, and the other enables memory retrospection through asynchronous traversals of the same location by multiple vehicles. We conduct experiments in place recog-nition and neural reconstruction. More importantly, MARS introduces new research opportunities and challenges such as multitraversal 3D reconstruction, multiagent perception, and unsupervised object discovery. Our data and codes can be found at https://ai4ce.github.io/MARS/.
ISSN:2575-7075
DOI:10.1109/CVPR52733.2024.02081