Marginal effects for non-linear prediction functions

• Published in: Data mining and knowledge discovery Vol. 38; no. 5; pp. 2997 - 3042
• Main Authors: Scholbeck, Christian A., Casalicchio, Giuseppe, Molnar, Christoph, Bischl, Bernd, Heumann, Christian
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.09.2024; Springer Nature B.V
• Subjects: Artificial Intelligence; Chemistry and Earth Sciences; Computer Science; Data Mining and Knowledge Discovery; Generalized linear models; Information Storage and Retrieval; Linear prediction; Machine learning; Nonlinearity; Physics; Regression models; Special Issue on Explainable and Interpretable Machine Learning and Data Mining; Statistical models; Statistics for Engineering; Subgroups; XAI; Interpretations; Model-agnostic; Interpretable machine learning; Conditional average marginal effects; IML; NLM; Non-linearity measure; Forward marginal effects; Explainable AI; Forward difference; cAME; FME
• ISSN: 1384-5810; 1573-756X
• DOI: 10.1007/s10618-023-00993-x

Beta coefficients for linear regression models represent the ideal form of an interpretable feature effect. However, for non-linear models such as generalized linear models, the estimated coefficients cannot be interpreted as a direct feature effect on the predicted outcome. Hence, marginal effects are typically used as approximations for feature effects, either as derivatives of the prediction function or forward differences in prediction due to changes in feature values. While marginal effects are commonly used in many scientific fields, they have not yet been adopted as a general model-agnostic interpretation method for machine learning models. This may stem from the ambiguity surrounding marginal effects and their inability to deal with the non-linearities found in black box models. We introduce a unified definition of forward marginal effects (FMEs) that includes univariate and multivariate, as well as continuous, categorical, and mixed-type features. To account for the non-linearity of prediction functions, we introduce a non-linearity measure for FMEs. Furthermore, we argue against summarizing feature effects of a non-linear prediction function in a single metric such as the average marginal effect. Instead, we propose to average homogeneous FMEs within population subgroups, which serve as conditional feature effect estimates.