Multi-View Depth Estimation by Using Adaptive Point Graph to Fuse Single-View Depth Probabilities

• Published in: IEEE robotics and automation letters Vol. 9; no. 7; pp. 6400 - 6407
• Main Authors: Wang, Ke, Liu, Chuhao, Liu, Zhanwen, Xiao, Fangwei, An, Yisheng, Zhao, Xiangmo, Shen, Shaojie
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.07.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Computer vision for automation; Costs; Datasets; Estimation; Fuses; Geometric constraints; Magnetic resonance imaging; Pixels; Probabilistic logic; range sensing; Three-dimensional displays; Uncertainty; visual learning and deep learning for visual perception
• ISSN: 2377-3766; 2377-3766
• DOI: 10.1109/LRA.2024.3405332

Recently, some methods estimate depth maps by fusing several adjacent single-view depth probabilities. They have achieved promising performance in multi-view inconsistent areas, such as texture-less surfaces, reflective surfaces, and moving objects. However, these methods involve two new problems: their thin cost volumes contain many invalid values, and the depths of adjacent volume units tend to be very different, which hinders the valid fusion of multi-view information. To deal with these issues, we design a novel point graph based single-views fusing method to estimate depth maps from several sequential images. Our method first estimates the initial probabilistic distribution of the depth map for input images, the distribution is parameterized as a pixel-wise depth and uncertainty. Then, we sample non-uniform depth candidates from the reference image's initial distribution. Diverse from the popular 3D cost volume, we utilize sampled depth candidates to construct an adaptive local point graph to represent multi-view geometric constraints. For pixels with multi-view consistency, we aggregate their local graphs to update their initial depths. And take the updated pixels as control points to refine the depth of the remaining pixels. We demonstrate the effectiveness of the proposed method by quantitative and qualitative comparisons with recent baseline works on the KITTI Odometry dataset and the DADD dataset, and our results surpass all competing methods even without 3D cost volume.