Generality Is Predictive of Prediction Accuracy

• Published in: Data Mining Vol. 3755; pp. 1 - 13
• Main Authors: Webb, Geoffrey I, Brain, Damien
• Format: Book Chapter
• Language: English
• Published: Berlin, Heidelberg Springer Berlin Heidelberg
• Series: Lecture Notes in Computer Science
• Subjects: Default Rule; Knowledge Acquisition; Prediction Accuracy; Training Data; Unseen Data
• ISBN: 3540325476; 9783540325475
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/11677437_1

During knowledge acquisition it frequently occurs that multiple alternative potential rules all appear equally credible. This paper addresses the dearth of formal analysis about how to select between such alternatives. It presents two hypotheses about the expected impact of selecting between classification rules of differing levels of generality in the absence of other evidence about their likely relative performance on unseen data. We argue that the accuracy on unseen data of the more general rule will tend to be closer to that of a default rule for the class than will that of the more specific rule. We also argue that in comparison to the more general rule, the accuracy of the more specific rule on unseen cases will tend to be closer to the accuracy obtained on training data. Experimental evidence is provided in support of these hypotheses. These hypotheses can be useful for selecting between rules in order to achieve specific knowledge acquisition objectives.