Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features

• Published in: Computational statistics Vol. 37; no. 5; pp. 2671 - 2692
• Main Authors: Pargent, Florian, Pfisterer, Florian, Thomas, Janek, Bischl, Bernd
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.11.2022; Springer Nature B.V
• Subjects: Algorithms; Best practice; Data analysis; Economic Theory/Quantitative Economics/Mathematical Methods; Machine learning; Mathematics and Statistics; Original Paper; Performance prediction; Probability and Statistics in Computer Science; Probability Theory and Stochastic Processes; Statistics; Support vector machines; Benchmark; Dummy encoding; Supervised machine learning; Generalized linear mixed models; Target encoding; High-cardinality categorical features

Since most machine learning (ML) algorithms are designed for numerical inputs, efficiently encoding categorical variables is a crucial aspect in data analysis. A common problem are high cardinality features, i.e. unordered categorical predictor variables with a high number of levels. We study techni...