Decision tree boosted varying coefficient models

• Published in: Data mining and knowledge discovery Vol. 36; no. 6; pp. 2237 - 2271
• Main Authors: Zhou, Yichen, Hooker, Giles
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.11.2022; Springer Nature B.V
• Subjects: Algorithms; Artificial Intelligence; Chemistry and Earth Sciences; Coefficients; Computer Science; Data Mining and Knowledge Discovery; Decision theory; Decision trees; Information Storage and Retrieval; Intelligibility; Parametric statistics; Physics; Spline functions; Statistics for Engineering; Boosting; Varying coefficient models; Decision trees; Semi-parametric models; Linear outputs
• ISSN: 1384-5810; 1573-756X
• DOI: 10.1007/s10618-022-00863-y

Varying coefficient models are a flexible extension of generic parametric models whose coefficients are functions of a set of effect-modifying covariates instead of fitted constants. They are capable of achieving higher model complexity while preserving the structure of the underlying parametric models, hence generating interpretable predictions. In this paper we study the use of gradient boosted decision trees as those coefficient-deciding functions in varying coefficient models with linearly structured outputs. In contrast to the traditional choices of splines or kernel smoothers, boosted trees are more flexible since they require no structural assumptions in the effect modifier space. We introduce our proposed method from the perspective of a localized version of gradient descent, prove its theoretical consistency under mild assumptions commonly adapted by decision tree research, and empirically demonstrate that the proposed tree boosted varying coefficient models achieve high performance qualified by their training speed, prediction accuracy and intelligibility as compared to several benchmark algorithms.