Beat the MTurkers: Automatic Image Labeling from Weak 3D Supervision

• Published in: 2014 IEEE Conference on Computer Vision and Pattern Recognition pp. 3198 - 3205
• Main Authors: Liang-Chieh Chen, Fidler, Sanja, Yuille, Alan L., Urtasun, Raquel
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2014
• Subjects: 3D vision; Computational modeling; crowd-sourcing; Design automation; Image segmentation; Laser radar; Object segmentation; Solid modeling; Three-dimensional displays; Training
• DOI: 10.1109/CVPR.2014.409
• ISSN: 1063-6919

Labeling large-scale datasets with very accurate object segmentations is an elaborate task that requires a high degree of quality control and a budget of tens or hundreds of thousands of dollars. Thus, developing solutions that can automatically perform the labeling given only weak supervision is key to reduce this cost. In this paper, we show how to exploit 3D information to automatically generate very accurate object segmentations given annotated 3D bounding boxes. We formulate the problem as the one of inference in a binary Markov random field which exploits appearance models, stereo and/or noisy point clouds, a repository of 3D CAD models as well as topological constraints. We demonstrate the effectiveness of our approach in the context of autonomous driving, and show that we can segment cars with the accuracy of 86% intersection-over-union, performing as well as highly recommended MTurkers!