Photorealistic Image Synthesis for Object Instance Detection

We present an approach to synthesize highly photorealistic images of 3D object models, which we use to train a convolutional neural network for detecting the objects in real images. The proposed approach has three key ingredients: (1) 3D object models are rendered in 3D models of complete scenes wit...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Hodan, Tomas, Vibhav Vineet, Gal, Ran, Shalev, Emanuel, Hanzelka, Jon, Connell, Treb, Urbina, Pedro, Sinha, Sudipta N, Guenter, Brian
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 09.02.2019
Subjects
Annotations
Artificial neural networks
Computer simulation
Computer vision
Ground truth
Image detection
Object recognition
Rendering
Synthesis
Three dimensional models
Websites
Online AccessGet full text

Cover

More Information
Summary:We present an approach to synthesize highly photorealistic images of 3D object models, which we use to train a convolutional neural network for detecting the objects in real images. The proposed approach has three key ingredients: (1) 3D object models are rendered in 3D models of complete scenes with realistic materials and lighting, (2) plausible geometric configuration of objects and cameras in a scene is generated using physics simulations, and (3) high photorealism of the synthesized images achieved by physically based rendering. When trained on images synthesized by the proposed approach, the Faster R-CNN object detector achieves a 24% absolute improvement of mAP@.75IoU on Rutgers APC and 11% on LineMod-Occluded datasets, compared to a baseline where the training images are synthesized by rendering object models on top of random photographs. This work is a step towards being able to effectively train object detectors without capturing or annotating any real images. A dataset of 600K synthetic images with ground truth annotations for various computer vision tasks will be released on the project website: thodan.github.io/objectsynth.
ISSN:2331-8422