A dynamic ensemble learning algorithm for neural networks

• Published in: Neural computing & applications Vol. 32; no. 12; pp. 8675 - 8690
• Main Authors: Alam, Kazi Md. Rokibul, Siddique, Nazmul, Adeli, Hojjat
• Format: Journal Article
• Language: English
• Published: London Springer London 01.06.2020; Springer Nature B.V
• Subjects: Algorithms; Artificial Intelligence; Computational Biology/Bioinformatics; Computational Science and Engineering; Computer Science; Data Mining and Knowledge Discovery; Ensemble learning; Heart diseases; Image Processing and Computer Vision; Machine learning; Mushrooms; Neural networks; Original Article; Probability and Statistics in Computer Science; Pruning; Soybeans; Backpropagation algorithm; Pruning algorithms; Neural network ensemble; Constructive algorithms; Negative correlation learning
• ISSN: 0941-0643; 1433-3058
• DOI: 10.1007/s00521-019-04359-7

This paper presents a novel dynamic ensemble learning (DEL) algorithm for designing ensemble of neural networks (NNs). DEL algorithm determines the size of ensemble, the number of individual NNs employing a constructive strategy, the number of hidden nodes of individual NNs employing a constructive–pruning strategy, and different training samples for individual NN’s learning. For diversity, negative correlation learning has been introduced and also variation of training samples has been made for individual NNs that provide better learning from the whole training samples. The major benefits of the proposed DEL compared to existing ensemble algorithms are (1) automatic design of ensemble; (2) maintaining accuracy and diversity of NNs at the same time; and (3) minimum number of parameters to be defined by user. DEL algorithm is applied to a set of real-world classification problems such as the cancer, diabetes, heart disease, thyroid, credit card, glass, gene, horse, letter recognition, mushroom, and soybean datasets. It has been confirmed by experimental results that DEL produces dynamic NN ensembles of appropriate architecture and diversity that demonstrate good generalization ability.