Gradual Training Method for Denoising Auto Encoders

Stacked denoising auto encoders (DAEs) are well known to learn useful deep representations, which can be used to improve supervised training by initializing a deep network. We investigate a training scheme of a deep DAE, where DAE layers are gradually added and keep adapting as additional layers are...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Kalmanovich, Alexander, Chechik, Gal
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 11.04.2015
Subjects
Coders
Datasets
Noise reduction
Training
Online AccessGet full text

Cover

More Information
Summary:Stacked denoising auto encoders (DAEs) are well known to learn useful deep representations, which can be used to improve supervised training by initializing a deep network. We investigate a training scheme of a deep DAE, where DAE layers are gradually added and keep adapting as additional layers are added. We show that in the regime of mid-sized datasets, this gradual training provides a small but consistent improvement over stacked training in both reconstruction quality and classification error over stacked training on MNIST and CIFAR datasets.
ISSN:2331-8422