An Evolutionary Algorithm for Learning Interpretable Ensembles of Classifiers

• Published in: Intelligent Systems Vol. 12319; pp. 18 - 33
• Main Authors: Cagnini, Henry E. L., Freitas, Alex A., Barros, Rodrigo C.
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2020; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Classification; Ensemble learning; Evolutionary algorithms; Machine learning; Supervised learning
• ISBN: 9783030613761; 3030613763
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-61377-8_2

Ensembles of classifiers are a very popular type of method for performing classification, due to their usually high predictive accuracy. However, ensembles have two drawbacks. First, ensembles are usually considered a ‘black box’, non-interpretable type of classification model, mainly because typically there are a very large number of classifiers in the ensemble (and often each classifier in the ensemble is a black-box classifier by itself). This lack of interpretability is an important limitation in application domains where a model’s predictions should be carefully interpreted by users, like medicine, law, etc. Second, ensemble methods typically involve many hyper-parameters, and it is difficult for users to select the best settings for those hyper-parameters. In this work we propose an Evolutionary Algorithm (an Estimation of Distribution Algorithm) that addresses both these drawbacks. This algorithm optimizes the hyper-parameter settings of a small ensemble of 5 interpretable classifiers, which allows users to interpret each classifier. In our experiments, the ensembles learned by the proposed Evolutionary Algorithm achieved the same level of predictive accuracy as a well-known Random Forest ensemble, but with the benefit of learning interpretable models (unlike Random Forests).