To Select or To Weigh: A Comparative Study of Linear Combination Schemes for SuperParent-One-Dependence Estimators

• Published in: IEEE transactions on knowledge and data engineering Vol. 19; no. 12; pp. 1652 - 1665
• Main Authors: Ying Yang, Webb, G.I., Cerquides, J., Korb, K.B., Boughton, J., Kai Ming Ting
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.12.2007; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Aprenentatge automàtic; Australia; Bayes methods; Bayesian methods; Benchmark testing; Benchmarking; Classification; Classification algorithms; Complexitat computacional; Computational complexity; Error analysis; Estadística bayesiana; Estimation theory; Estimators; Information technology; Large-scale systems; Learning; Learning (artificial intelligence); Learning systems; Machine learning; Pattern classification; Performance analysis; Performance evaluation of algorithms and systems; Reconeixement de formes (Informàtica); Senior members; Statistical testing; Statistical tests; Teoria de l'estimació; Terminology; Weighing; Performance evaluation of algorithms and systems; Machine learning
• ISSN: 1041-4347; 1558-2191
• DOI: 10.1109/TKDE.2007.190650

We conduct a large-scale comparative study on linearly combining superparent-one-dependence estimators (SPODEs), a popular family of seminaive Bayesian classifiers. Altogether, 16 model selection and weighing schemes, 58 benchmark data sets, and various statistical tests are employed. This paper's main contributions are threefold. First, it formally presents each scheme's definition, rationale, and time complexity and hence can serve as a comprehensive reference for researchers interested in ensemble learning. Second, it offers bias-variance analysis for each scheme's classification error performance. Third, it identifies effective schemes that meet various needs in practice. This leads to accurate and fast classification algorithms which have an immediate and significant impact on real-world applications. Another important feature of our study is using a variety of statistical tests to evaluate multiple learning methods across multiple data sets.