In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning

We present experiments demonstrating that some other form of capacity control, different from network size, plays a central role in learning multilayer feed-forward networks. We argue, partially through analogy to matrix factorization, that this is an inductive bias that can help shed light on deep...

Full description

Saved in:
Bibliographic Details
Main Authors Neyshabur, Behnam, Tomioka, Ryota, Srebro, Nathan
Format Journal Article
LanguageEnglish
Published 20.12.2014
Subjects
Computer Science - Artificial Intelligence
Computer Science - Computer Vision and Pattern Recognition
Computer Science - Learning
Statistics - Machine Learning
Online AccessGet full text

Cover

More Information
Summary:We present experiments demonstrating that some other form of capacity control, different from network size, plays a central role in learning multilayer feed-forward networks. We argue, partially through analogy to matrix factorization, that this is an inductive bias that can help shed light on deep learning.
DOI:10.48550/arxiv.1412.6614