Data Distillation: Towards Omni-Supervised Learning

We investigate omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data. Omni-supervised learning is lower-bounded by performance on existing labeled datasets, offering the potential...

Full description

Saved in:
Bibliographic Details
Published in2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition pp. 4119 - 4128
Main Authors Radosavovic, Ilija, Dollar, Piotr, Girshick, Ross, Gkioxari, Georgia, He, Kaiming
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2018
Subjects
Data models
Head
Heating systems
Predictive models
Semisupervised learning
Training
Transforms
Online AccessGet full text

Cover

More Information
Summary:We investigate omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data. Omni-supervised learning is lower-bounded by performance on existing labeled datasets, offering the potential to surpass state-of-the-art fully supervised methods. To exploit the omni-supervised setting, we propose data distillation, a method that ensembles predictions from multiple transformations of unlabeled data, using a single model, to automatically generate new training annotations. We argue that visual recognition models have recently become accurate enough that it is now possible to apply classic ideas about self-training to challenging real-world data. Our experimental results show that in the cases of human keypoint detection and general object detection, state-of-the-art models trained with data distillation surpass the performance of using labeled data from the COCO dataset alone.
ISSN:1063-6919
DOI:10.1109/CVPR.2018.00433