Generalization and Expressivity for Deep Nets

• Published in: IEEE transaction on neural networks and learning systems Vol. 30; no. 5; pp. 1392 - 1406
• Main Author: Lin, Shao-Bo
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.05.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Approximation; Biological neural networks; Deep learning; expressivity; generalization; Learning systems; Learning theory; localized approximation; Machine learning; Mathematical analysis; Nets; Neurons; Optimization; Power measurement; Risk management; Sparse representation

Along with the rapid development of deep learning in practice, theoretical explanations for its success become urgent. Generalization and expressivity are two widely used measurements to quantify theoretical behaviors of deep nets. The expressivity focuses on finding functions expressible by deep ne...