Limitations of shallow nets approximation

In this paper, we aim at analyzing the approximation abilities of shallow networks in reproducing kernel Hilbert spaces (RKHSs). We prove that there is a probability measure such that the achievable lower bound for approximating by shallow nets can be realized for all functions in balls of reproduci...

Full description

Saved in:
Bibliographic Details
Published inNeural networks Vol. 94; pp. 96 - 102
Main Author Lin, Shao-Bo
Format Journal Article
LanguageEnglish
Published United States Elsevier Ltd 01.10.2017
Subjects
Approximation
Deep nets
Neural Networks (Computer)
Probability
Reproducing kernel Hilbert space
Shallow nets
Reproducing kernel Hilbert space
Approximation
Deep nets
Shallow nets
Online AccessGet full text

Cover

More Information
Summary:In this paper, we aim at analyzing the approximation abilities of shallow networks in reproducing kernel Hilbert spaces (RKHSs). We prove that there is a probability measure such that the achievable lower bound for approximating by shallow nets can be realized for all functions in balls of reproducing kernel Hilbert space with high probability, which is different with the classical minimax approximation error estimates. This result together with the existing approximation results for deep nets shows the limitations for shallow nets and provides a theoretical explanation on why deep nets perform better than shallow nets.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:0893-6080
1879-2782
DOI:10.1016/j.neunet.2017.06.016