Cross Modal Distillation for Supervision Transfer

In this work we propose a technique that transfers supervision between images from different modalities. We use learned representations from a large labeled modality as supervisory signal for training representations for a new unlabeled paired modality. Our method enables learning of rich representa...

Full description

Saved in:
Bibliographic Details
Published in2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 2827 - 2836
Main Authors Gupta, Saurabh, Hoffman, Judy, Malik, Jitendra
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2016
Subjects
Adaptation models
Computer architecture
Image representation
Optical imaging
Optical sensors
Semantics
Training
Online AccessGet full text

Cover

More Information
Summary:In this work we propose a technique that transfers supervision between images from different modalities. We use learned representations from a large labeled modality as supervisory signal for training representations for a new unlabeled paired modality. Our method enables learning of rich representations for unlabeled modalities and can be used as a pre-training procedure for new modalities with limited labeled data. We transfer supervision from labeled RGB images to unlabeled depth and optical flow images and demonstrate large improvements for both these cross modal supervision transfers.
ISSN:1063-6919
DOI:10.1109/CVPR.2016.309