Event-Based Stereo Visual Odometry

• Published in: IEEE transactions on robotics Vol. 37; no. 5; pp. 1433 - 1450
• Main Authors: Zhou, Yi, Gallego, Guillermo, Shen, Shaojie
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.10.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Cameras; Computer vision; Consistency; Data acquisition; Datasets; Dynamic range; Linear programming; Modules; Real time operation; Real-time systems; Representations; Robot vision systems; Robotics; Simultaneous localization and mapping; smart cameras; stereo vision; Tracking
• ISSN: 1552-3098; 1941-0468
• DOI: 10.1109/TRO.2021.3062252

Event-based cameras are bioinspired vision sensors whose pixels work independently from each other and respond asynchronously to brightness changes, with microsecond resolution. Their advantages make it possible to tackle challenging scenarios in robotics, such as high-speed and high dynamic range scenes. We present a solution to the problem of visual odometry from the data acquired by a stereo event-based camera rig. Our system follows a parallel tracking-and-mapping approach, where novel solutions to each subproblem (three-dimensional (3-D) reconstruction and camera pose estimation) are developed with two objectives in mind: being principled and efficient, for real-time operation with commodity hardware. To this end, we seek to maximize the spatio-temporal consistency of stereo event-based data while using a simple and efficient representation. Specifically, the mapping module builds a semidense 3-D map of the scene by fusing depth estimates from multiple viewpoints (obtained by spatio-temporal consistency) in a probabilistic fashion. The tracking module recovers the pose of the stereo rig by solving a registration problem that naturally arises due to the chosen map and event data representation. Experiments on publicly available datasets and on our own recordings demonstrate the versatility of the proposed method in natural scenes with general 6-DoF motion. The system successfully leverages the advantages of event-based cameras to perform visual odometry in challenging illumination conditions, such as low-light and high dynamic range, while running in real-time on a standard CPU. We release the software and dataset under an open source license to foster research in the emerging topic of event-based simultaneous localization and mapping.