Depth Estimation and Semantic Segmentation from a Single RGB Image Using a Hybrid Convolutional Neural Network

• Published in: Sensors (Basel, Switzerland) Vol. 19; no. 8; p. 1795
• Main Authors: Lin, Xiao, Sánchez-Escobedo, Dalila, Casas, Josep R, Pardàs, Montse
• Format: Journal Article Publication
• Language: English
• Published: Switzerland Multidisciplinary Digital Publishing Institute (MDPI) 15.04.2019; MDPI; MDPI AG
• Subjects: convolutional neural networks; depth estimation; Enginyeria de la telecomunicació; hybrid architecture; Image processing; Imatges; Neural networks (Computer science); Processament; semantic segmentation; Xarxes neuronals (Informàtica); Àrees temàtiques de la UPC; hybrid architecture; convolutional neural networks; semantic segmentation; depth estimation
• ISSN: 1424-8220; 1424-8220
• DOI: 10.3390/s19081795

Semantic segmentation and depth estimation are two important tasks in computer vision, and many methods have been developed to tackle them. Commonly these two tasks are addressed independently, but recently the idea of merging these two problems into a sole framework has been studied under the assumption that integrating two highly correlated tasks may benefit each other to improve the estimation accuracy. In this paper, depth estimation and semantic segmentation are jointly addressed using a single RGB input image under a unified convolutional neural network. We analyze two different architectures to evaluate which features are more relevant when shared by the two tasks and which features should be kept separated to achieve a mutual improvement. Likewise, our approaches are evaluated under two different scenarios designed to review our results versus single-task and multi-task methods. Qualitative and quantitative experiments demonstrate that the performance of our methodology outperforms the state of the art on single-task approaches, while obtaining competitive results compared with other multi-task methods.