Can local explanation techniques explain linear additive models?

• Published in: Data mining and knowledge discovery Vol. 38; no. 1; pp. 237 - 280
• Main Authors: Rahnama, Amir Hossein Akhavan, Bütepage, Judith, Geurts, Pierre, Boström, Henrik
• Format: Journal Article Web Resource
• Language: English
• Published: New York Springer US 01.01.2024; Springer Nature B.V; Springer
• Subjects: Accuracy; Additive models; Additives; Artificial Intelligence; Black box modelling; Black boxes; Chemistry and Earth Sciences; Computer Networks and Communications; Computer Science; Computer Science Applications; Data Mining and Knowledge Discovery; Datalogi; Datasets; Engineering, computing & technology; Explainable machine learning; Information Storage and Retrieval; Information Systems; Ingénierie, informatique & technologie; LIME; Local model; Local model-agnostic explanation; Local model-agnostic explanations; Local permutations; Logistics regressions; Machine-learning; Mathematical models; Non-additive; Performance prediction; Permutations; Physics; S.I. : ECML PKDD 2023; Sciences informatiques; SHAP; Statistics for Engineering; SHAP; LIME; Local model-agnostic explanations; Explainable machine learning
• DOI: 10.1007/s10618-023-00971-3
• ISSN: 1384-5810; 1573-756X

Local model-agnostic additive explanation techniques decompose the predicted output of a black-box model into additive feature importance scores. Questions have been raised about the accuracy of the produced local additive explanations. We investigate this by studying whether some of the most popular explanation techniques can accurately explain the decisions of linear additive models. We show that even though the explanations generated by these techniques are linear additives, they can fail to provide accurate explanations when explaining linear additive models. In the experiments, we measure the accuracy of additive explanations, as produced by, e.g., LIME and SHAP, along with the non-additive explanations of Local Permutation Importance (LPI) when explaining Linear and Logistic Regression and Gaussian naive Bayes models over 40 tabular datasets. We also investigate the degree to which different factors, such as the number of numerical or categorical or correlated features, the predictive performance of the black-box model, explanation sample size, similarity metric, and the pre-processing technique used on the dataset can directly affect the accuracy of local explanations.