EPP-MVSNet: Epipolar-assembling based Depth Prediction for Multi-view Stereo

• Published in: Proceedings / IEEE International Conference on Computer Vision pp. 5712 - 5720
• Main Authors: Ma, Xinjun, Gong, Yue, Wang, Qirui, Huang, Jingwei, Chen, Lei, Yu, Fan
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.10.2021
• Subjects: 3D from a single image and shape-from-x; 3D from multiview and other sensors; Costs; Efficient training and inference methods; Image resolution; Learning systems; Memory management; Redundancy; Refining; Stereo; Three-dimensional displays
• DOI: 10.1109/ICCV48922.2021.00568

In this paper, we proposed EPP-MVSNet, a novel deep learning network for 3D reconstruction from multi-view stereo (MVS). EPP-MVSNet can accurately aggregate features at high resolution to a limited cost volume with an optimal depth range, thus, leads to effective and efficient 3D construction. Distinct from existing works which measure feature cost at discrete positions which affects the 3D reconstruction accuracy, EPP-MVSNet introduces an epipolar-assembling-based kernel that operates on adaptive intervals along epipolar lines for making full use of the image resolution. Further, we introduce an entropy-based refining strategy where the cost volume describes the space geometry with the little redundancy. Moreover, we design a light-weighted network with Pseudo-3D convolutions integrated to achieve high accuracy and efficiency. We have conducted extensive experiments on challenging datasets Tanks & Temples(TNT), ETH3D and DTU. As a result, we achieve promising results on all datasets and the highest F-Score on the online TNT intermediate benchmark. Code is available at https://gitee.com/mindspore/mindspore/tree/master/model_zoo/research/cv/eppmvsnet.