Event-aided Direct Sparse Odometry
We introduce EDS, a direct monocular visual odometry using events and frames. Our algorithm leverages the event generation model to track the camera motion in the blind time between frames. The method formulates a direct probabilistic approach of observed brightness increments. Per-pixel brightness...
Saved in:
Published in | 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 5771 - 5780 |
---|---|
Main Authors | , , |
Format | Conference Proceeding |
Language | English |
Published |
IEEE
01.06.2022
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | We introduce EDS, a direct monocular visual odometry using events and frames. Our algorithm leverages the event generation model to track the camera motion in the blind time between frames. The method formulates a direct probabilistic approach of observed brightness increments. Per-pixel brightness increments are predicted using a sparse number of selected 3D points and are compared to the events via the brightness increment error to estimate camera motion. The method recovers a semi-dense 3D map using photometric bundle adjustment. EDS is the first method to perform 6-DOF VO using events and frames with a direct approach. By design it overcomes the problem of changing appearance in indirect methods. Our results outperform all previous event-based odometry solutions. We also show that, for a target error performance, EDS can work at lower frame rates than state-of-the-art frame-based VO solutions. This opens the door to low-power motion-tracking applications where frames are sparingly triggered "on demand" and our method tracks the motion in between. We release code and datasets to the public. |
---|---|
AbstractList | We introduce EDS, a direct monocular visual odometry using events and frames. Our algorithm leverages the event generation model to track the camera motion in the blind time between frames. The method formulates a direct probabilistic approach of observed brightness increments. Per-pixel brightness increments are predicted using a sparse number of selected 3D points and are compared to the events via the brightness increment error to estimate camera motion. The method recovers a semi-dense 3D map using photometric bundle adjustment. EDS is the first method to perform 6-DOF VO using events and frames with a direct approach. By design it overcomes the problem of changing appearance in indirect methods. Our results outperform all previous event-based odometry solutions. We also show that, for a target error performance, EDS can work at lower frame rates than state-of-the-art frame-based VO solutions. This opens the door to low-power motion-tracking applications where frames are sparingly triggered "on demand" and our method tracks the motion in between. We release code and datasets to the public. |
Author | Hidalgo-Carrio, Javier Scaramuzza, Davide Gallego, Guillermo |
Author_xml | – sequence: 1 givenname: Javier surname: Hidalgo-Carrio fullname: Hidalgo-Carrio, Javier organization: Univ. of Zurich,Dept. of Informatics – sequence: 2 givenname: Guillermo surname: Gallego fullname: Gallego, Guillermo organization: Technische Universität Berlin, Einstein Center Digital Future and SCIoI Excellence Cluster,Germany – sequence: 3 givenname: Davide surname: Scaramuzza fullname: Scaramuzza, Davide organization: Univ. of Zurich,Dept. of Informatics |
BookMark | eNotzstKAzEUgOEoCra1T6CLwf2MJyeXkyxlrBcoVLxtSyZzAgE7LZlB6Nsr6OrfffxzcTbsBxbiWkIjJfjb9vPl1aB1rkFAbACM9SdiLq012npt1amYoSFTE5C5EMtxzB0YBCDl3UzcrL55mOqQe-6r-1w4TtXbIZSRq02_3_FUjpfiPIWvkZf_XYiPh9V7-1SvN4_P7d26zghqqjFpQzr4EDoJXfw9UsAREgUXtUuJMQYER0FS9BSp7xIkTtH2rFEpoxbi6s_NzLw9lLwL5bj1jrxzUv0AxvRBnA |
CODEN | IEEPAD |
ContentType | Conference Proceeding |
DBID | 6IE 6IH CBEJK RIE RIO |
DOI | 10.1109/CVPR52688.2022.00569 |
DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE IEEE Proceedings Order Plans (POP) 1998-present |
DatabaseTitleList | |
Database_xml | – sequence: 1 dbid: RIE name: IEEE url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/ sourceTypes: Publisher |
DeliveryMethod | fulltext_linktorsrc |
EISBN | 1665469463 9781665469463 |
EISSN | 2575-7075 |
EndPage | 5780 |
ExternalDocumentID | 9879881 |
Genre | orig-research |
GrantInformation_xml | – fundername: European Research Council (ERC) grantid: 864042 funderid: 10.13039/100010663 – fundername: Swiss National Science Foundation grantid: 51NF40_185543 funderid: 10.13039/501100001711 |
GroupedDBID | 6IE 6IH 6IL 6IN ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP OCL RIE RIL RIO |
ID | FETCH-LOGICAL-i203t-2f4574a9aab10bc68830ec0f7a8c48ffe2ca2087a17c97c7dbf0fefc6de423353 |
IEDL.DBID | RIE |
IngestDate | Wed Jun 26 19:28:28 EDT 2024 |
IsPeerReviewed | false |
IsScholarly | true |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-i203t-2f4574a9aab10bc68830ec0f7a8c48ffe2ca2087a17c97c7dbf0fefc6de423353 |
PageCount | 10 |
ParticipantIDs | ieee_primary_9879881 |
PublicationCentury | 2000 |
PublicationDate | 2022-June |
PublicationDateYYYYMMDD | 2022-06-01 |
PublicationDate_xml | – month: 06 year: 2022 text: 2022-June |
PublicationDecade | 2020 |
PublicationTitle | 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) |
PublicationTitleAbbrev | CVPR |
PublicationYear | 2022 |
Publisher | IEEE |
Publisher_xml | – name: IEEE |
SSID | ssib052007398 ssib042469789 |
Score | 2.3922412 |
Snippet | We introduce EDS, a direct monocular visual odometry using events and frames. Our algorithm leverages the event generation model to track the camera motion in... |
SourceID | ieee |
SourceType | Publisher |
StartPage | 5771 |
SubjectTerms | Brightness Bundle adjustment Codes Low-level vision; Motion and tracking; Robot vision Sensitivity Target tracking Technological innovation Tracking loops |
Title | Event-aided Direct Sparse Odometry |
URI | https://ieeexplore.ieee.org/document/9879881 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LSwMxEA61J08qrfhmEY-mzSbZTXIuLUWoFrXSW8ljAiK2pewe6q832e1DxIO3kFMmr5lJvu8bhO5SQsDk3GAV_B_mygLWuWHYsZA7hIDdOhK5w6PHfDjhD9Ns2kD3Oy4MAFTgM-jEZvWX7xa2jE9l3ZAfKxl51geS0Jqrtd07nIY874dyelQTEkzJDVsuJarbexs_R3GTCOiiUaYzq1DO-5oqlUsZHKHRdjA1kuSjUxamY79-6TT-d7THqL0n7yXjnVs6QQ2Yt9BtP-IacdSDdEl9zSUvy5DUQvLkFp9QrNZtNBn0X3tDvCmPgN8pYQWmnmeCa6W1SYmxwTBGwBIvtLRceg_Uakqk0KmwSljhjCcevM0dhBiKZewUNeeLOZyhxJtMgc-9pjIcauakcs5HFhB1XJjcnKNWtG-2rBUwZhvTLv7uvkSHcYZrQNUVaharEq6D6y7MTbVm34G1l4c |
link.rule.ids | 310,311,786,790,795,796,802,23958,23959,25170,27958,55109 |
linkProvider | IEEE |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LTwIxEG4IHvSkBoxvN8ajhdJ2t9szgaACEgXDjfQxTQwRCFkO-uttd3kY48Fbs-lhZl_fzO73fYPQXYMQ0AnXWHr8w1wawCrRDFvmewdfsBtLgna41086I_44jscldL_VwgBATj6DWljm__Lt3KzCp7K6749lGnTWex7niSzUWpu7h1Pf6f3wTg9-QoLJdK2X8_vrzbfBS7A3CZQuGow645znvJuqkoNK-xD1NuEUXJJpbZXpmvn65dT433iPUHUn34sGW2A6RiWYVdBtKzAbcXCEtFHxooteF76thejZzj8gW35W0ajdGjY7eD0gAb9TwjJMHY8FV1Ip3SDa-MQYAUOcUKnhqXNAjaIkFaohjBRGWO2IA2cSC76KYjE7QeXZfAanKHI6luASp2jqH2tmU2mtCzogarnQiT5DlZDfZFF4YEzWqZ3_ffgG7XeGve6k-9B_ukAH4WwX9KpLVM6WK7jyQJ7p6_z6fQO2dprd |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=2022+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29&rft.atitle=Event-aided+Direct+Sparse+Odometry&rft.au=Hidalgo-Carrio%2C+Javier&rft.au=Gallego%2C+Guillermo&rft.au=Scaramuzza%2C+Davide&rft.date=2022-06-01&rft.pub=IEEE&rft.eissn=2575-7075&rft.spage=5771&rft.epage=5780&rft_id=info:doi/10.1109%2FCVPR52688.2022.00569&rft.externalDocID=9879881 |