Provable approximation properties for deep neural networks

We discuss approximation of functions using deep neural nets. Given a function f on a d-dimensional manifold Γ⊂Rm, we construct a sparsely-connected depth-4 neural network and bound its error in approximating f. The size of the network depends on dimension and curvature of the manifold Γ, the comple...

Full description

Saved in:
Bibliographic Details
Published inApplied and computational harmonic analysis Vol. 44; no. 3; pp. 537 - 557
Main Authors Shaham, Uri, Cloninger, Alexander, Coifman, Ronald R.
Format Journal Article
LanguageEnglish
Published Elsevier Inc 01.05.2018
Subjects
Function approximation
Neural nets
Wavelets
Neural nets
Function approximation
Wavelets
Online AccessGet full text

Cover

More Information
Summary:We discuss approximation of functions using deep neural nets. Given a function f on a d-dimensional manifold Γ⊂Rm, we construct a sparsely-connected depth-4 neural network and bound its error in approximating f. The size of the network depends on dimension and curvature of the manifold Γ, the complexity of f, in terms of its wavelet description, and only weakly on the ambient dimension m. Essentially, our network computes wavelet functions, which are computed from Rectified Linear Units (ReLU).
ISSN:1063-5203
1096-603X
DOI:10.1016/j.acha.2016.04.003