Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Prediction

We present a framework for learning single-view shape and pose prediction without using direct supervision for either. Our approach allows leveraging multi-view observations from unknown poses as supervisory signal during training. Our proposed training setup enforces geometric consistency between t...

Full description

Saved in:
Bibliographic Details
Published in2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition pp. 2897 - 2905
Main Authors Tulsiani, Shubham, Efros, Alexei A., Malik, Jitendra
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2018
Subjects
Cameras
Geometry
Image reconstruction
Loss measurement
Shape
Three-dimensional displays
Training
Online AccessGet full text
ISSN1063-6919
DOI10.1109/CVPR.2018.00306

Cover

More Information
Summary:We present a framework for learning single-view shape and pose prediction without using direct supervision for either. Our approach allows leveraging multi-view observations from unknown poses as supervisory signal during training. Our proposed training setup enforces geometric consistency between the independently predicted shape and pose from two views of the same instance. We consequently learn to predict shape in an emergent canonical (view-agnostic) frame along with a corresponding pose predictor. We show empirical and qualitative results using the ShapeNet dataset and observe encouragingly competitive performance to previous techniques which rely on stronger forms of supervision. We also demonstrate the applicability of our framework in a realistic setting which is beyond the scope of existing techniques: using a training dataset comprised of online product images where the underlying shape and pose are unknown.
ISSN:1063-6919
DOI:10.1109/CVPR.2018.00306