Cross-Stitch Networks for Multi-task Learning

Multi-task learning in Convolutional Networks has displayed remarkable success in the field of recognition. This success can be largely attributed to learning shared representations from multiple supervisory tasks. However, existing multi-task approaches rely on enumerating multiple network architec...

Full description

Saved in:
Bibliographic Details
Published in2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 3994 - 4003
Main Authors Misra, Ishan, Shrivastava, Abhinav, Gupta, Abhinav, Hebert, Martial
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2016
Subjects
Computer architecture
Computer vision
Estimation
Face
Network architecture
Semantics
Training
Online AccessGet full text

Cover

More Information
Summary:Multi-task learning in Convolutional Networks has displayed remarkable success in the field of recognition. This success can be largely attributed to learning shared representations from multiple supervisory tasks. However, existing multi-task approaches rely on enumerating multiple network architectures specific to the tasks at hand, that do not generalize. In this paper, we propose a principled approach to learn shared representations in ConvNets using multitask learning. Specifically, we propose a new sharing unit: "cross-stitch" unit. These units combine the activations from multiple networks and can be trained end-to-end. A network with cross-stitch units can learn an optimal combination of shared and task-specific representations. Our proposed method generalizes across multiple tasks and shows dramatically improved performance over baseline methods for categories with few training examples.
ISSN:1063-6919
DOI:10.1109/CVPR.2016.433