Robust and Efficient Boosting Method Using the Conditional Risk

• Published in: IEEE transaction on neural networks and learning systems Vol. 29; no. 7; pp. 3069 - 3083
• Main Authors: Xiao, Zhi, Luo, Zhe, Zhong, Bo, Dang, Xin
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.07.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: AdaBoosting; Bayesian analysis; Boosting; classification; Competitiveness; conditional risk; exponential loss; Heuristic algorithms; label noise; Learning algorithms; Machine learning; Noise sensitivity; overfitting; Risk; Robustness; Training; Trustworthiness; Uncertainty
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2017.2711028

Well known for its simplicity and effectiveness in classification, AdaBoost, however, suffers from overfitting when class-conditional distributions have significant overlap. Moreover, it is very sensitive to noise that appears in the labels. This paper tackles the above limitations simultaneously via optimizing a modified loss function (i.e., the conditional risk). The proposed approach has the following two advantages. First, it is able to directly take into account label uncertainty with an associated label confidence. Second, it introduces a trustworthiness measure on training samples via the Bayesian risk rule, and hence the resulting classifier tends to have finite sample performance that is superior to that of the original AdaBoost when there is a large overlap between class conditional distributions. Theoretical properties of the proposed method are investigated. Extensive experimental results using synthetic data and real-world data sets from UCI machine learning repository are provided. The empirical study shows the high competitiveness of the proposed method in predication accuracy and robustness when compared with the original AdaBoost and several existing robust AdaBoost algorithms.