Explainable models via compression of tree ensembles

Ensemble models (bagging and gradient-boosting) of relational decision trees have proved to be some of the most effective learning methods in the area of probabilistic logic models (PLMs). While effective, they lose one of the most important benefits of PLMs—interpretability. In this paper we consid...

Full description

Saved in:
Bibliographic Details
Published inMachine learning Vol. 113; no. 3; pp. 1303 - 1328
Main Authors Yan, Siwen, Natarajan, Sriraam, Joshi, Saket, Khardon, Roni, Tadepalli, Prasad
Format Journal Article
LanguageEnglish
Published New York Springer US 01.03.2024
Springer Nature B.V
Subjects
Artificial Intelligence
Compressing
Computer Science
Control
Decision trees
Logic
Machine Learning
Mechatronics
Natural Language Processing (NLP)
Neural networks
Robotics
Semantics
Simulation and Modeling
Special Issue on Learning and Reasoning 2022
Variables
TILDE
Tree combination
First order logic
Ensemble
Online AccessGet full text

Cover

More Information
Summary:Ensemble models (bagging and gradient-boosting) of relational decision trees have proved to be some of the most effective learning methods in the area of probabilistic logic models (PLMs). While effective, they lose one of the most important benefits of PLMs—interpretability. In this paper we consider the problem of compressing a large set of learned trees into a single explainable model. To this effect, we propose CoTE—Compression of Tree Ensembles—that produces a single small decision list as a compressed representation. CoTE first converts the trees to decision lists and then performs the combination and compression with the aid of the original training set. An experimental evaluation demonstrates the effectiveness of CoTE in several benchmark relational data sets.
ISSN:0885-6125
1573-0565
DOI:10.1007/s10994-023-06463-1