Multiagent Multitraversal Multimodal Self-Driving: Open MARS Dataset
Large-scale datasets have fueled recent advancements in AI-based autonomous vehicle research. However, these datasets are usually collected from a single vehicle's one-time pass of a certain location, lacking multiagent inter-actions or repeated traversals of the same place. Such information co...
Saved in:
Published in | 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 22041 - 22051 |
---|---|
Main Authors | , , , , , , , |
Format | Conference Proceeding |
Language | English |
Published |
IEEE
16.06.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Large-scale datasets have fueled recent advancements in AI-based autonomous vehicle research. However, these datasets are usually collected from a single vehicle's one-time pass of a certain location, lacking multiagent inter-actions or repeated traversals of the same place. Such information could lead to transformative enhancements in autonomous vehicles' perception, prediction, and planning capabilities. To bridge this gap, in collaboration with the self-driving company May Mobility, we present MARS dataset which unifies scenarios that enable MultiAgent, multitraveRSal, and multimodal autonomous vehicle re-search. More specifically, MARS is collected with a fleet of autonomous vehicles driving within a certain geographical area. Each vehicle has its own route and different vehicles may appear at nearby locations. Each vehicle is equipped with a LiDAR and surround-view RGB cameras. We curate two subsets in MARS: one facilitates collaborative driving with multiple vehicles simultaneously present at the same location, and the other enables memory retrospection through asynchronous traversals of the same location by multiple vehicles. We conduct experiments in place recog-nition and neural reconstruction. More importantly, MARS introduces new research opportunities and challenges such as multitraversal 3D reconstruction, multiagent perception, and unsupervised object discovery. Our data and codes can be found at https://ai4ce.github.io/MARS/. |
---|---|
AbstractList | Large-scale datasets have fueled recent advancements in AI-based autonomous vehicle research. However, these datasets are usually collected from a single vehicle's one-time pass of a certain location, lacking multiagent inter-actions or repeated traversals of the same place. Such information could lead to transformative enhancements in autonomous vehicles' perception, prediction, and planning capabilities. To bridge this gap, in collaboration with the self-driving company May Mobility, we present MARS dataset which unifies scenarios that enable MultiAgent, multitraveRSal, and multimodal autonomous vehicle re-search. More specifically, MARS is collected with a fleet of autonomous vehicles driving within a certain geographical area. Each vehicle has its own route and different vehicles may appear at nearby locations. Each vehicle is equipped with a LiDAR and surround-view RGB cameras. We curate two subsets in MARS: one facilitates collaborative driving with multiple vehicles simultaneously present at the same location, and the other enables memory retrospection through asynchronous traversals of the same location by multiple vehicles. We conduct experiments in place recog-nition and neural reconstruction. More importantly, MARS introduces new research opportunities and challenges such as multitraversal 3D reconstruction, multiagent perception, and unsupervised object discovery. Our data and codes can be found at https://ai4ce.github.io/MARS/. |
Author | Li, Yiming Feng, Chen Gong, Moonjun Jiang, Peili Li, Zhiheng Chen, Nuo Lyu, Zonglin Wang, Zehong |
Author_xml | – sequence: 1 givenname: Yiming surname: Li fullname: Li, Yiming email: yimingli@nyu.edu organization: New York University – sequence: 2 givenname: Zhiheng surname: Li fullname: Li, Zhiheng organization: New York University – sequence: 3 givenname: Nuo surname: Chen fullname: Chen, Nuo organization: New York University – sequence: 4 givenname: Moonjun surname: Gong fullname: Gong, Moonjun organization: New York University – sequence: 5 givenname: Zonglin surname: Lyu fullname: Lyu, Zonglin organization: New York University – sequence: 6 givenname: Zehong surname: Wang fullname: Wang, Zehong organization: New York University – sequence: 7 givenname: Peili surname: Jiang fullname: Jiang, Peili organization: New York University – sequence: 8 givenname: Chen surname: Feng fullname: Feng, Chen email: cfeng@nyu.edu organization: New York University |
BookMark | eNotjs1OwkAURkejiYh9AxZ9gdY7_zPuSCtqAsEAcUuunVsyphTSVhLfXiKuzjmbL989u2kPLTE24ZBzDv6x-HhfaWGlzAUIlYMAx69Y4q13UoPUEsBcs5HQVmcWrL5jSd9_AYAUnBvvRqxcfDdDxB21Q_qnQ4cn6npsLrk_hLOuqamzsoun2O6e0uWR2nQxXa3TEgfsaXhgtzU2PSX_HLPN7HlTvGbz5ctbMZ1n0VieWQnWgKswoPZKKWe0r4VU6hNMbUiYCtALCtX5HYJAH5wyoUaPyH0gJcdscpmNRLQ9dnGP3c-Wg9EOnJW_LcFNJg |
CODEN | IEEPAD |
ContentType | Conference Proceeding |
DBID | 6IE 6IH CBEJK RIE RIO |
DOI | 10.1109/CVPR52733.2024.02081 |
DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE/IET Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
DatabaseTitleList | |
Database_xml | – sequence: 1 dbid: RIE name: IEEE/IET Electronic Library (IEL) url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/ sourceTypes: Publisher |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Applied Sciences |
EISBN | 9798350353006 |
EISSN | 2575-7075 |
EndPage | 22051 |
ExternalDocumentID | 10658087 |
Genre | orig-research |
GroupedDBID | 6IE 6IH 6IL 6IN ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP OCL RIE RIL RIO |
ID | FETCH-LOGICAL-i671-7307608cada594448659f2344b06f6e26c0a92edc032a02a9d846dfa9aa19de43 |
IEDL.DBID | RIE |
IngestDate | Wed Sep 25 09:21:55 EDT 2024 |
IsDoiOpenAccess | false |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-i671-7307608cada594448659f2344b06f6e26c0a92edc032a02a9d846dfa9aa19de43 |
OpenAccessLink | http://arxiv.org/pdf/2406.09383 |
PageCount | 11 |
ParticipantIDs | ieee_primary_10658087 |
PublicationCentury | 2000 |
PublicationDate | 2024-June-16 |
PublicationDateYYYYMMDD | 2024-06-16 |
PublicationDate_xml | – month: 06 year: 2024 text: 2024-June-16 day: 16 |
PublicationDecade | 2020 |
PublicationTitle | 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) |
PublicationTitleAbbrev | CVPR |
PublicationYear | 2024 |
Publisher | IEEE |
Publisher_xml | – name: IEEE |
SSID | ssj0003211698 |
Score | 1.9290681 |
Snippet | Large-scale datasets have fueled recent advancements in AI-based autonomous vehicle research. However, these datasets are usually collected from a single... |
SourceID | ieee |
SourceType | Publisher |
StartPage | 22041 |
SubjectTerms | Autonomous Driving Collaboration Computer vision Data collection Dataset and Benchmark Laser radar Mars Neural Reconstruction Semantics Three-dimensional displays |
Title | Multiagent Multitraversal Multimodal Self-Driving: Open MARS Dataset |
URI | https://ieeexplore.ieee.org/document/10658087 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LawIxEA7VU0_2YembPfS622Q3j01vRStSUERt8SazeYDUukXXS399k93V0kKhtySXhAnDl5nMNx9Cd2AUyxgVoQCAkKZceZkXCLMk05YlkhjmucODIe-_0OcZm9Vk9ZILY4wpi89M5IflX77O1danypyHO7zEqWigRorjiqy1T6gkLpThMq3pcQTL-87raOz7iyUuDIxp5OUoyQ8RlRJDei003O1elY68Rdsii9Tnr8aM_z7eEWp_0_WC0R6IjtGBWZ2gVv2-DGrv3Zyibsm2BU-mCirirdceWm9gWU3fc-2GE7O0YXe98JmGh8AXnASDx_Ek6ELhEK9oo2nvadrph7WKQrjggoTOgwXHqQINTFIXjHEmbZxQmmFuuYm5wiBjo5UzIOAYpHYvEm1BAhCpDU3OUHOVr8w5CjRXNNWKYWMJtUBS6-JbQRUI6kBf6QvU9kaZf1R9MuY7e1z-sX6FDv3F-MIrwq9Rs1hvzY2D-CK7La_2C8zTpfE |
link.rule.ids | 310,311,786,790,795,796,802,27956,55107 |
linkProvider | IEEE |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LTwIxEG4UD3rCB8a3e_C6a3e3j603AxJUIATQcCOzfSREBAPLxV9vu7tgNDHx1vbSZprJ15nONx9CN6AlTSnhPgcAnyRMOpkX8NM4VYbGItTUcYc7XdZ6IU8jOirJ6jkXRmudF5_pwA3zv3w1lyuXKrMebvESJ3wb7Vigx7yga21SKrENZphISoJciMVt_bXXdx3GYhsIRiRwgpThDxmVHEWaVdRd718Uj7wFqywN5Oev1oz_PuA-qn0T9rzeBooO0JaeHaJq-cL0Sv9dHqFGzrcFR6fyCuqtUx9aLGFaTN_nyg4Hemr8xmLicg13nis58Tr3_YHXgMxiXlZDw-bDsN7ySx0Ff8J46Fsf5gwnEhRQQWw4xqgwUUxIiplhOmISg4i0ktaAgCMQyr5JlAEBEAqlSXyMKrP5TJ8gTzFJEiUp1iYkBsLE2AiXEwmcWNiX6hTVnFHGH0WnjPHaHmd_rF-j3daw0x63H7vP52jPXZIrwwrZBapki5W-tICfpVf5NX8BvO2pRQ |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2024+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29&rft.atitle=Multiagent+Multitraversal+Multimodal+Self-Driving%3A+Open+MARS+Dataset&rft.au=Li%2C+Yiming&rft.au=Li%2C+Zhiheng&rft.au=Chen%2C+Nuo&rft.au=Gong%2C+Moonjun&rft.date=2024-06-16&rft.pub=IEEE&rft.eissn=2575-7075&rft.spage=22041&rft.epage=22051&rft_id=info:doi/10.1109%2FCVPR52733.2024.02081&rft.externalDocID=10658087 |