Robustified L 2 boosting

• Published in: Computational statistics & data analysis Vol. 52; no. 7; pp. 3331 - 3341
• Main Authors: Lutz, Roman Werner, Kalisch, Markus, Bühlmann, Peter
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 15.03.2008
• ISSN: 0167-9473; 1872-7352
• DOI: 10.1016/j.csda.2007.11.006

Five robustifications of L 2 boosting for linear regression with various robustness properties are considered. The first two use the Huber loss as implementing loss function for boosting and the second two use robust simple linear regression for the fitting in L 2 boosting (i.e. robust base learners). Both concepts can be applied with or without down-weighting of leverage points. Our last method uses robust correlation estimates and appears to be most robust. Crucial advantages of all methods are that they do not compute covariance matrices of all covariates and that they do not have to identify multivariate leverage points. When there are no outliers, the robust methods are only slightly worse than L 2 boosting. In the contaminated case though, the robust methods outperform L 2 boosting by a large margin. Some of the robustifications are also computationally highly efficient and therefore well suited for truly high-dimensional problems.