Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation

Convolutional neural networks are designed for dense data, but vision data is often sparse (stereo depth, point clouds, pen stroke, etc.). We present a method to handle sparse depth data with optional dense RGB, and accomplish depth completion and semantic segmentation changing only the last layer....

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Jaritz, Maximilian, de Charette, Raoul, Wirbel, Emilie, Perrotton, Xavier, Nashashibi, Fawzi
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 31.08.2018
Subjects
Artificial neural networks
Lidar
Neural networks
Semantic segmentation
Semantics
Online AccessGet full text

Cover

More Information
Summary:Convolutional neural networks are designed for dense data, but vision data is often sparse (stereo depth, point clouds, pen stroke, etc.). We present a method to handle sparse depth data with optional dense RGB, and accomplish depth completion and semantic segmentation changing only the last layer. Our proposal efficiently learns sparse features without the need of an additional validity mask. We show how to ensure network robustness to varying input sparsities. Our method even works with densities as low as 0.8% (8 layer lidar), and outperforms all published state-of-the-art on the Kitti depth completion benchmark.
ISSN:2331-8422