NICE: an algorithm for nearest instance counterfactual explanations

• Published in: Data mining and knowledge discovery Vol. 38; no. 5; pp. 2665 - 2703
• Main Authors: Brughmans, Dieter, Leyman, Pieter, Martens, David
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.09.2024; Springer Nature B.V
• Subjects: Algorithms; Artificial Intelligence; Chemistry and Earth Sciences; Computer Science; Data Mining and Knowledge Discovery; Information Storage and Retrieval; Optimization; Physics; Search process; Sparsity; Special Issue on Explainable and Interpretable Machine Learning and Data Mining; Statistics for Engineering; XAI; Counterfactual explanations; Machine learning
• ISSN: 1384-5810; 1573-756X
• DOI: 10.1007/s10618-023-00930-y

In this paper we propose a new algorithm, named NICE, to generate counterfactual explanations for tabular data that specifically takes into account algorithmic requirements that often emerge in real-life deployments: (1) the ability to provide an explanation for all predictions, (2) being able to handle any classification model (also non-differentiable ones), (3) being efficient in run time, and (4) providing multiple counterfactual explanations with different characteristics. More specifically, our approach exploits information from a nearest unlike neighbor to speed up the search process, by iteratively introducing feature values from this neighbor in the instance to be explained. We propose four versions of NICE, one without optimization and, three which optimize the explanations for one of the following properties: sparsity, proximity or plausibility. An extensive empirical comparison on 40 datasets shows that our algorithm outperforms the current state-of-the-art in terms of these criteria. Our analyses show a trade-off between on the one hand plausibility and on the other hand proximity or sparsity, with our different optimization methods offering users the choice to select the types of counterfactuals that they prefer. An open-source implementation of NICE can be found at https://github.com/ADMAntwerp/NICE.