Dropout vs. batch normalization: an empirical study of their impact to deep learning

• Published in: Multimedia tools and applications Vol. 79; no. 19-20; pp. 12777 - 12815
• Main Authors: Garbin, Christian, Zhu, Xingquan, Marques, Oge
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.05.2020; Springer Nature B.V
• Subjects: Artificial neural networks; Computer Communication Networks; Computer programming; Computer Science; Data Structures and Information Theory; Deep learning; Experimentation; Machine learning; Model accuracy; Multimedia Information Systems; Neural networks; Smartphones; Special Purpose and Application-Based Systems; Training; Tuning; Deep learning; Regularization; Batch normalization; Optimization; Overfitting; Dropout

Overfitting and long training time are two fundamental challenges in multilayered neural network learning and deep learning in particular. Dropout and batch normalization are two well-recognized approaches to tackle these challenges. While both approaches share overlapping design principles, numerou...