Multiple Regularization and Analysis of Deep Capsule Network

With the increase of layers in deep capsule networks, the overfitting problem also becomes more serious. Capsule-based regularization methods are important to solve this problem. However, little attention has been paid to this field. To fill this gap, we propose five regularization methods from the...

Full description

Saved in:
Bibliographic Details
Published inPattern analysis and applications : PAA Vol. 25; no. 4; pp. 711 - 729
Main Authors Sun, Kun, Xu, Haixia, Yuan, Liming, Wen, Xianbin
Format Journal Article
LanguageEnglish
Published London Springer London 01.11.2022
Springer Nature B.V
Subjects
Computer Science
Image reconstruction
Pattern Recognition
Regularization
Tensors
Theoretical Advances
Training
Capsule networks
Image recognition
Deep neural networks
Regularization
Online AccessGet full text

Cover

More Information
Summary:With the increase of layers in deep capsule networks, the overfitting problem also becomes more serious. Capsule-based regularization methods are important to solve this problem. However, little attention has been paid to this field. To fill this gap, we propose five regularization methods from the following aspects. In capsules represented by vectors, two methods are proposed to modify the existence and properties of their activation vectors by disturbing the length and orientation of the vectors. In capsules represented by tensors, capsule-based layer normalization is proposed to improve dynamic routing. In the training strategy, a warm restart learning rate with probability is used to improve the efficiency of training. In reconstruction, a novel image decoder provides a better regularization effect by using multiscale information of images. These regularization methods are investigated on CIFAR10, CIFAR100, and SVHN. Experiments show that using these regularization methods can effectively improve the generalization performance.
ISSN:1433-7541
1433-755X
DOI:10.1007/s10044-022-01070-7