Deep Sparse-coded Network (DSN)

We present Deep Sparse-coded Network (DSN), a deep architecture based on multilayer sparse coding. It has been considered difficult to learn a useful feature hierarchy by stacking sparse coding layers in a straightforward manner. The primary reason is the modeling assumption for sparse coding that t...

Full description

Saved in:
Bibliographic Details
Published in2016 23rd International Conference on Pattern Recognition (ICPR) pp. 2610 - 2615
Main Authors Youngjune Gwon, Cha, Miriam, Kung, H. T.
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.12.2016
Subjects
Backpropagation
Computer architecture
Dictionaries
Encoding
Neural networks
Nonhomogeneous media
Training
Online AccessGet full text
DOI10.1109/ICPR.2016.7900029

Cover

More Information
Summary:We present Deep Sparse-coded Network (DSN), a deep architecture based on multilayer sparse coding. It has been considered difficult to learn a useful feature hierarchy by stacking sparse coding layers in a straightforward manner. The primary reason is the modeling assumption for sparse coding that takes in a dense input and yields a sparse output vector. Applying a sparse coding layer on the output of another tends to violate the modeling assumption. We overcome this shortcoming by interlacing nonlinear pooling units. Average- or max-pooled sparse codes are aggregated to form dense input vectors for the next sparse coding layer. Pooling achieves nonlinear activation analogous to neural networks while not introducing diminished gradient flows during the training. We introduce a novel backpropagation algorithm to finetune the proposed DSN beyond the pretraining via greedy layerwise sparse coding and dictionary learning. We build an experimental 4-layer DSN with the ℓ 1 -regularized LARS and the greedy-ℓ 0 OMP, and demonstrate superior performance over a similarly-configured stacked autoencoder (SAE) on CIFAR-10.
DOI:10.1109/ICPR.2016.7900029