RBoost: Label Noise-Robust Boosting Algorithm Based on a Nonconvex Loss Function and the Numerically Stable Base Learners

• Published in: IEEE transaction on neural networks and learning systems Vol. 27; no. 11; pp. 2216 - 2228
• Main Authors: Miao, Qiguang, Cao, Ying, Xia, Ge, Gong, Maoguo, Liu, Jiachen, Song, Jianfeng
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.11.2016
• Subjects: AdaBoost; adaptive Newton step; Algorithm design and analysis; Algorithms; antinoise; Boosting; Estimation; loss function; Machine Learning; Neural Networks (Computer); Noise; Noise measurement; Robustness; Training
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2015.2475750

AdaBoost has attracted much attention in the machine learning community because of its excellent performance in combining weak classifiers into strong classifiers. However, AdaBoost tends to overfit to the noisy data in many applications. Accordingly, improving the antinoise ability of AdaBoost plays an important role in many applications. The sensitiveness to the noisy data of AdaBoost stems from the exponential loss function, which puts unrestricted penalties to the misclassified samples with very large margins. In this paper, we propose two boosting algorithms, referred to as RBoost1 and RBoost2, which are more robust to the noisy data compared with AdaBoost. RBoost1 and RBoost2 optimize a nonconvex loss function of the classification margin. Because the penalties to the misclassified samples are restricted to an amount less than one, RBoost1 and RBoost2 do not overfocus on the samples that are always misclassified by the previous base learners. Besides the loss function, at each boosting iteration, RBoost1 and RBoost2 use numerically stable ways to compute the base learners. These two improvements contribute to the robustness of the proposed algorithms to the noisy training and testing samples. Experimental results on the synthetic Gaussian data set, the UCI data sets, and a real malware behavior data set illustrate that the proposed RBoost1 and RBoost2 algorithms perform better when the training data sets contain noisy data.