End-To-End Multi-Task Learning With Attention

We propose a novel multi-task learning architecture, which allows learning of task-specific feature-level attention. Our design, the Multi-Task Attention Network (MTAN), consists of a single shared network containing a global feature pool, together with a soft-attention module for each task. These m...

Full description

Saved in:
Bibliographic Details
Published in2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 1871 - 1880
Main Authors Liu, Shikun, Johns, Edward, Davison, Andrew J.
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2019
Subjects
Computer architecture
Computer vision
Deep Learning
Grouping and Shape
Multitasking
Network architecture
Neural networks
Representation Learning
Robustness
Segmentation
Visualization
Online AccessGet full text

Cover

More Information
Summary:We propose a novel multi-task learning architecture, which allows learning of task-specific feature-level attention. Our design, the Multi-Task Attention Network (MTAN), consists of a single shared network containing a global feature pool, together with a soft-attention module for each task. These modules allow for learning of task-specific features from the global features, whilst simultaneously allowing for features to be shared across different tasks. The architecture can be trained end-to-end and can be built upon any feed-forward neural network, is simple to implement, and is parameter efficient. We evaluate our approach on a variety of datasets, across both image-to-image predictions and image classification tasks. We show that our architecture is state-of-the-art in multi-task learning compared to existing methods, and is also less sensitive to various weighting schemes in the multi-task loss function. Code is available at https://github.com/lorenmt/mtan.
ISSN:2575-7075
DOI:10.1109/CVPR.2019.00197