Parallax Inference for Robust Temporal Monocular Depth Estimation in Unstructured Environments

• Published in: Sensors (Basel, Switzerland) Vol. 22; no. 23; p. 9374
• Main Authors: Fonder, Michaël, Ernst, Damien, Van Droogenbroeck, Marc
• Format: Journal Article Web Resource
• Language: English
• Published: Switzerland MDPI AG 01.12.2022; Multidisciplinary Digital Publishing Institute (MDPI); MDPI
• Subjects: Autonomous vehicles; Cameras; Computer science; Datasets; deep learning; depth estimation; Driverless cars; drone; Engineering, computing & technology; Estimates; Estimation; Ingénierie, informatique & technologie; KITTI; Methods; Mid-air; Motion; Motion Perception; Neural networks; Neural Networks, Computer; Parallax; RGB camera; Sciences informatiques; Sensors; UAV; unmanned aerial vehicle; unmanned vehicles; deep learning; parallax; unmanned vehicles; depth estimation
• ISSN: 1424-8220; 1424-3210; 1424-8220
• DOI: 10.3390/s22239374

Estimating the distance to objects is crucial for autonomous vehicles, but cost, weight or power constraints sometimes prevent the use of dedicated depth sensors. In this case, the distance has to be estimated from on-board mounted RGB cameras, which is a complex task especially for environments such as natural outdoor landscapes. In this paper, we present a new depth estimation method suitable for use in such landscapes. First, we establish a bijective relationship between depth and the visual parallax of two consecutive frames and show how to exploit it to perform motion-invariant pixel-wise depth estimation. Then, we detail our architecture which is based on a pyramidal convolutional neural network where each level refines an input parallax map estimate by using two customized cost volumes. We use these cost volumes to leverage the visual spatio-temporal constraints imposed by motion and make the network robust for varied scenes. We benchmarked our approach both in test and generalization modes on public datasets featuring synthetic camera trajectories recorded in a wide variety of outdoor scenes. Results show that our network outperforms the state of the art on these datasets, while also performing well on a standard depth estimation benchmark.