Learning Not to Learn: Training Deep Neural Networks with Biased Data
We propose a novel regularization algorithm to train deep neural networks, in which data at training time is severely biased. Since a neural network efficiently learns data distribution, a network is likely to learn the bias information to categorize input data. It leads to poor performance at test...
Saved in:
Main Authors | , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
26.12.2018
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | We propose a novel regularization algorithm to train deep neural networks, in
which data at training time is severely biased. Since a neural network
efficiently learns data distribution, a network is likely to learn the bias
information to categorize input data. It leads to poor performance at test
time, if the bias is, in fact, irrelevant to the categorization. In this paper,
we formulate a regularization loss based on mutual information between feature
embedding and bias. Based on the idea of minimizing this mutual information, we
propose an iterative algorithm to unlearn the bias information. We employ an
additional network to predict the bias distribution and train the network
adversarially against the feature embedding network. At the end of learning,
the bias prediction network is not able to predict the bias not because it is
poorly trained, but because the feature embedding network successfully unlearns
the bias information. We also demonstrate quantitative and qualitative
experimental results which show that our algorithm effectively removes the bias
information from feature embedding. |
---|---|
DOI: | 10.48550/arxiv.1812.10352 |