Learning label smoothing for text classification

• Published in: PeerJ. Computer science Vol. 10; p. e2005
• Main Authors: Ren, Han, Zhao, Yajie, Zhang, Yong, Sun, Wei
• Format: Journal Article
• Language: English
• Published: United States PeerJ. Ltd 23.04.2024; PeerJ Inc
• Subjects: Algorithms and Analysis of Algorithms; Analysis; Computational Linguistics; Data Mining and Machine Learning; Excessive regularization; Label smoothing; Language processing; Natural language interfaces; Neural network; Neural Networks; Soft label; Text classification; Text Mining; New York; Neural network; Label smoothing; Excessive regularization; Soft label; Text classification
• ISSN: 2376-5992; 2376-5992
• DOI: 10.7717/peerj-cs.2005

Training with soft labels instead of hard labels can effectively improve the robustness and generalization of deep learning models. Label smoothing often provides uniformly distributed soft labels during the training process, whereas it does not take the semantic difference of labels into account. This article introduces discrimination-aware label smoothing, an adaptive label smoothing approach that learns appropriate distributions of labels for iterative optimization objectives. In this approach, positive and negative samples are employed to provide experience from both sides, and the performances of regularization and model calibration are improved through an iterative learning method. Experiments on five text classification datasets demonstrate the effectiveness of the proposed method.