Three-teaching: A three-way decision framework to handle noisy labels

• Published in: Applied soft computing Vol. 154; p. 111400
• Main Authors: Chao, Guoqing, Zhang, Kaiwen, Wang, Xiru, Chu, Dianhui
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.03.2024
• Subjects: Deep neural network; Noisy labels; Three-teaching; Noisy labels; Deep neural network; Three-teaching
• ISSN: 1568-4946; 1872-9681
• DOI: 10.1016/j.asoc.2024.111400

Learning with noisy labels represents a prevalent weakly supervised learning paradigm. Uncertain knowledge resulting from noisy labels poses significant challenges for knowledge analysis. Given the memorization effect observed in deep neural networks, training on instances with minimal loss holds promise for effectively handling noisy labels. “Co-teaching”, which is the state-of-the-art training method in this field, is characterized by the simultaneous training of two deep neural networks using instances with low loss. While this approach has demonstrated promising performance, its effectiveness heavily relies on the predictive capabilities of two neural networks. If these networks fail to provide reliable predictions, the overall learning performance may be unsatisfactory. In order to solve this problem and inspired by three-way decision, we propose a powerful learning paradigm named “Three-teaching”, which employs the “voting mechanism” to guarantee the prediction quality incrementally. In this approach, both neural networks make predictions for all the data. However, only the data that exhibits consistent prediction results and has a low loss is retained to feed into the third neural network for updating its parameters. The learning process will proceed by alternating these three neural networks’ roles. The experimental results obtained from benchmark datasets illustrate that “Three-teaching” surpasses numerous state-of-the-art methods. •We proposed a three-teaching model which introduces another neural network and “voting mechanism” to guarantee the prediction quality while facing noisy labels.•Three-teaching performs better than co-teaching and other compared methods on several real-world datasets.•Three-teaching provides a new way to implement three-way decision.