A novel reasoning mechanism for multi-label text classification

• Published in: Information processing & management Vol. 58; no. 2; p. 102441
• Main Authors: Wang, Ran, Ridley, Robert, Su, Xi’ao, Qu, Weiguang, Dai, Xinyu
• Format: Journal Article
• Language: English
• Published: Oxford Elsevier Ltd 01.03.2021; Elsevier Science Publishers; Elsevier Science Ltd
• Subjects: Algorithms; Analysis; Classification; Classifiers; Cognition & reasoning; Descriptive labeling; Iterative methods; Iterative reasoning mechanism; Label embedding; Labels; Machine learning; Multi-label learning; Reasoning; Text categorization; Text classification; 99-00; Multi-label learning; Label embedding; Iterative reasoning mechanism; 00-01; Text classification
• ISSN: 0306-4573; 1873-5371
• DOI: 10.1016/j.ipm.2020.102441

•A novel reasoning-based algorithm named ML-Reasoner for multi-label text classification is proposed.•Each instance of reasoning in this method takes the previously predicted likelihoods for all labels as additional input.•Not only is the method able to avoid the dependency of label orders completely, but it also achieves competitive performance when handling with multi-label datasets.•Applying the reasoning mechanism to three strong neural-based base models can achieve significant performance improvements on all two data sets.•The method achieves state-of-the-art results on two challenging multi-label datasets. The aim in multi-label text classification is to assign a set of labels to a given document. Previous classifier-chain and sequence-to-sequence models have been shown to have a powerful ability to capture label correlations. However, they rely heavily on the label order, while labels in multi-label data are essentially an unordered set. The performance of these approaches is therefore highly variable depending on the order in which the labels are arranged. To avoid being dependent on label order, we design a reasoning-based algorithm named Multi-Label Reasoner (ML-Reasoner) for multi-label classification. ML-Reasoner employs a binary classifier to predict all labels simultaneously and applies a novel iterative reasoning mechanism to effectively utilize the inter-label information, where each instance of reasoning takes the previously predicted likelihoods for all labels as additional input. This approach is able to utilize information between labels, while avoiding the issue of label-order sensitivity. Extensive experiments demonstrate that our method outperforms state-of-the art approaches on the challenging AAPD dataset. We also apply our reasoning module to a variety of strong neural-based base models and show that it is able to boost performance significantly in each case.