Simplified swarm optimization for hyperparameters of convolutional neural networks

• Published in: Computers & industrial engineering Vol. 177; p. 109076
• Main Authors: Yeh, Wei-Chang, Lin, Yi-Ping, Liang, Yun-Chia, Lai, Chyh-Ming, Huang, Chia-Ling
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.03.2023
• Subjects: Convolutional neural networks; Hyperparameter optimization; Image recognition; Machine learning; Simplified swarm optimization; Hyperparameter optimization; Image recognition; Convolutional neural networks; Machine learning; Simplified swarm optimization
• ISSN: 0360-8352; 1879-0550
• DOI: 10.1016/j.cie.2023.109076

•A hyperparameter optimization algorithm called SSO-LeNet is proposed to improve LeNet.•SSO-LeNet is the first SSO-based algorithm to improve LeNet.•SSO-LeNet outperforms LeNet in solution quality with a significant improvement in test time.•SSO-LeNet outperforms PSO-LeNet in solution quality.•SSO-LeNet has considerable potential for improving AlexNet, GoogLeNet, etc. efficiently. Convolutional neural networks (CNNs) are widely used in image recognition. Numerous CNN models, such as LeNet, AlexNet, VGG, ResNet, and GoogLeNet, have been developed by increasing the number of layers to improve performance. However, performance deteriorates beyond a certain number of layers. Hence, hyperparameter optimization is a more efficient way to improve CNNs. To validate this concept, in the present study, an algorithm based on simplified swarm optimization was developed for optimizing the hyperparameters of the simplest CNN model: LeNet. The results of experiments involving the MNIST, Fashion-MNIST, and CIFAR-10 datasets indicated that the accuracy of the proposed algorithm was higher than those of LeNet and PSO-LeNet and that the proposed algorithm can be applied to more complex models such as AlexNet.