GeoNet: Geometric Neural Network for Joint Depth and Surface Normal Estimation

In this paper, we propose Geometric Neural Network (GeoNet) to jointly predict depth and surface normal maps from a single image. Building on top of two-stream CNNs, our GeoNet incorporates geometric relation between depth and surface normal via the new depth-to-normal and normal-to-depth networks....

Full description

Saved in:
Bibliographic Details
Published in2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition pp. 283 - 291
Main Authors Qi, Xiaojuan, Liao, Renjie, Liu, Zhengzhe, Urtasun, Raquel, Jia, Jiaya
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2018
Subjects
Computer architecture
Estimation
Kernel
Neural networks
Rough surfaces
Surface roughness
Three-dimensional displays
Online AccessGet full text

Cover

More Information
Summary:In this paper, we propose Geometric Neural Network (GeoNet) to jointly predict depth and surface normal maps from a single image. Building on top of two-stream CNNs, our GeoNet incorporates geometric relation between depth and surface normal via the new depth-to-normal and normal-to-depth networks. Depth-to-normal network exploits the least square solution of surface normal from depth and improves its quality with a residual module. Normal-to-depth network, contrarily, refines the depth map based on the constraints from the surface normal through a kernel regression module, which has no parameter to learn. These two networks enforce the underlying model to efficiently predict depth and surface normal for high consistency and corresponding accuracy. Our experiments on NYU v2 dataset verify that our GeoNet is able to predict geometrically consistent depth and normal maps. It achieves top performance on surface normal estimation and is on par with state-of-the-art depth estimation methods.
ISSN:2575-7075
DOI:10.1109/CVPR.2018.00037