Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach

We present a theoretically grounded approach to train deep neural networks, including recurrent networks, subject to class-dependent label noise. We propose two procedures for loss correction that are agnostic to both application domain and network architecture. They simply amount to at most a matri...

Full description

Saved in:
Bibliographic Details
Published in2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 2233 - 2241
Main Authors Patrini, Giorgio, Rozza, Alessandro, Menon, Aditya Krishna, Nock, Richard, Lizhen Qu
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.07.2017
Subjects
Computer architecture
Computer vision
Neural networks
Noise measurement
Robustness
Training
Online AccessGet full text

Cover

More Information
Summary:We present a theoretically grounded approach to train deep neural networks, including recurrent networks, subject to class-dependent label noise. We propose two procedures for loss correction that are agnostic to both application domain and network architecture. They simply amount to at most a matrix inversion and multiplication, provided that we know the probability of each class being corrupted into another. We further show how one can estimate these probabilities, adapting a recent technique for noise estimation to the multi-class setting, and thus providing an end-to-end framework. Extensive experiments on MNIST, IMDB, CIFAR-10, CIFAR-100 and a large scale dataset of clothing images employing a diversity of architectures - stacking dense, convolutional, pooling, dropout, batch normalization, word embedding, LSTM and residual layers - demonstrate the noise robustness of our proposals. Incidentally, we also prove that, when ReLU is the only non-linearity, the loss curvature is immune to class-dependent label noise.
ISSN:1063-6919
1063-6919
DOI:10.1109/CVPR.2017.240