A hybrid scheme-based one-vs-all decision trees for multi-class classification tasks

• Published in: Knowledge-based systems Vol. 198; p. 105922
• Main Authors: Yan, Jianjian, Zhang, Zhongnan, Lin, Kunhui, Yang, Fan, Luo, Xiongbiao
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 21.06.2020; Elsevier Science Ltd
• Subjects: Algorithms; Classification; Decision tree; Decision trees; Hybrid scheme; Multi-class classification; One-vs-all; Permutations; Split criteria; Splitting; Hybrid scheme; Multi-class classification; Split criteria; Decision tree; One-vs-all
• ISSN: 0950-7051; 1872-7409
• DOI: 10.1016/j.knosys.2020.105922

Decision tree algorithms have been proved to be a powerful and popular approach in classification tasks. However, they do not have reasonable classification performance in multi-class scenarios. In the present study, decision tree algorithms are combined with the one-vs-all (OVA) binarization technique to improve the generalization capabilities of the scheme. However, unlike previous literature that has focused on aggregation strategies, the present study is focused on the process of building base classifiers over the OVA scheme. A novel split criterion, entitled by the splitting point correction matrix (SPCM), is proposed in this regards, which can effectively deal with the unbalance problem caused by the OVA scheme. The SPCM is a kind of hybrid scheme, which integrates distribution and permutation information from the training data at each splitting point. Therefore, compared to other classical split criteria, such as the C4.5, the proposed method can make the right choice about the optimal splitting point at the root or internal nodes from multi-angle. In order to evaluate the effectiveness of the SPCM approach, extensive experiments are carried out compared to the classical and state-of-the-art methods. The experiments are performed on sixteen datasets, where the effectiveness and the accuracy of the proposed method is verified. It is concluded that the SPCM method not only has excellent classification performance but also produces a more compact decision tree. Moreover, it is found that the SPCM method has especially a considerable improvement in the depth of tree. [Display omitted] •This is the first paper to deal with the class imbalanced problem caused by OVA strategy.•A novel split criterion is proposed to tackle the imbalanced problem caused by OVA strategy.•The proposed method selects the optimal splitting point from multi-angle.•The proposed method significantly reduces the depth of decision tree.