A new heuristic of the decision tree induction

• Published in: 2009 International Conference on Machine Learning and Cybernetics Vol. 3; pp. 1659 - 1664
• Main Authors: Ning Li, Li Zhao, Ai-Xia Chen, Qing-Wu Meng, Guo-Fang Zhang
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.07.2009
• Subjects: Classification tree analysis; Clustering; Cybernetics; Decision tree; Decision trees; Entropy; Generalization; Induction generators; Inverse problems; large margin; Machine learning; Machine learning algorithms; SMO; Support vector machine classification; Support vector machines
• ISBN: 9781424437023; 1424437024
• ISSN: 2160-133X
• DOI: 10.1109/ICMLC.2009.5212227

Decision tree induction is one of the useful approaches for extracting classification knowledge from a set of feature-based instances. The most popular heuristic information used in the decision tree generation is the minimum entropy. This heuristic information has a serious disadvantage-the poor generalization capability [3]. Support vector machine (SVM) is a classification technique of machine learning based on statistical learning theory. It has good generalization. Considering the relationship between the classification margin of support vector machine(SVM) and the generalization capability, the large margin of SVM can be used as the heuristic information of decision tree, in order to improve its generalization capability. This paper proposes a decision tree induction algorithm based on large margin heuristic. Comparing with the binary decision tree using the minimum entropy as the heuristic information, the experiments show that the generalization capability has been improved by using the new heuristic.