Max-Pooling Dropout for Regularization of Convolutional Neural Networks

• Published in: Neural Information Processing Vol. 9489; pp. 46 - 54
• Main Authors: Wu, Haibing, Gu, Xiaodong
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 01.01.2015; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Convolutional neural network; Deep learning; Max-pooling dropout; Pattern recognition
• ISBN: 3319265318; 9783319265315
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-26532-2_6

Recently, dropout has seen increasing use in deep learning. For deep convolutional neural networks, dropout is known to work well in fully-connected layers. However, its effect in pooling layers is still not clear. This paper demonstrates that max-pooling dropout is equivalent to randomly picking activation based on a multinomial distribution at training time. In light of this insight, we advocate employing our proposed probabilistic weighted pooling, instead of commonly used max-pooling, to act as model averaging at test time. Empirical evidence validates the superiority of probabilistic weighted pooling. We also compare max-pooling dropout and stochastic pooling, both of which introduce stochasticity based on multinomial distributions at pooling stage.