Inverting Visual Representations with Convolutional Networks

Feature representations, both hand-designed and learned ones, are often hard to analyze and interpret, even when they are extracted from visual data. We propose a new approach to study image representations by inverting them with an up-convolutional neural network. We apply the method to shallow rep...

Full description

Saved in:
Bibliographic Details
Published in2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 4829 - 4837
Main Authors Dosovitskiy, Alexey, Brox, Thomas
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2016
Subjects
Computer architecture
Feature extraction
Image color analysis
Image reconstruction
Microprocessors
Neural networks
Training
Online AccessGet full text

Cover

More Information
Summary:Feature representations, both hand-designed and learned ones, are often hard to analyze and interpret, even when they are extracted from visual data. We propose a new approach to study image representations by inverting them with an up-convolutional neural network. We apply the method to shallow representations (HOG, SIFT, LBP), as well as to deep networks. For shallow representations our approach provides significantly better reconstructions than existing methods, revealing that there is surprisingly rich information contained in these features. Inverting a deep network trained on ImageNet provides several insights into the properties of the feature representation learned by the network. Most strikingly, the colors and the rough contours of an image can be reconstructed from activations in higher network layers and even from the predicted class probabilities.
ISSN:1063-6919
DOI:10.1109/CVPR.2016.522