Depth and surface normal estimation from monocular images using regression on deep features and hierarchical CRFs

• Published in: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 1119 - 1127
• Main Authors: Bo Li, Chunhua Shen, Yuchao Dai, van den Hengel, Anton, Mingyi He
• Format: Conference Proceeding Journal Article
• Language: English
• Published: IEEE 01.06.2015
• Subjects: Color; Computer vision; Context; Estimation; Exact solutions; Feature extraction; Inference; Neural networks; Pattern recognition; Pixels; Regression; Three-dimensional displays; Training; Training data
• ISSN: 1063-6919; 1063-6919
• DOI: 10.1109/CVPR.2015.7298715

Predicting the depth (or surface normal) of a scene from single monocular color images is a challenging task. This paper tackles this challenging and essentially underdetermined problem by regression on deep convolutional neural network (DCNN) features, combined with a post-processing refining step using conditional random fields (CRF). Our framework works at two levels, super-pixel level and pixel level. First, we design a DCNN model to learn the mapping from multi-scale image patches to depth or surface normal values at the super-pixel level. Second, the estimated super-pixel depth or surface normal is refined to the pixel level by exploiting various potentials on the depth or surface normal map, which includes a data term, a smoothness term among super-pixels and an auto-regression term characterizing the local structure of the estimation map. The inference problem can be efficiently solved because it admits a closed-form solution. Experiments on the Make3D and NYU Depth V2 datasets show competitive results compared with recent state-of-the-art methods.