Automatic classifier selection for non-experts

Choosing a suitable classifier for a given dataset is an important part of developing a pattern recognition system. Since a large variety of classification algorithms are proposed in literature, non-experts do not know which method should be used in order to obtain good classification results on the...

Full description

Saved in:

Bibliographic Details
Published in	Pattern analysis and applications : PAA Vol. 17; no. 1; pp. 83 - 96
Main Authors	Reif, Matthias, Shafait, Faisal, Goldstein, Markus, Breuel, Thomas, Dengel, Andreas
Format	Journal Article
Language	English
Published	London Springer London 01.02.2014 Springer
Subjects	Algorithmics. Computability. Computer arithmetics Applied sciences Computer Science Computer science; control theory; systems Data processing. List processing. Character string processing Exact sciences and technology Memory organisation. Data processing Pattern Recognition Software Theoretical Advances Theoretical computing Regression Classifier recommendation Classifier selection Meta-features Meta-learning Landmarking High performance Data analysis Automatic classification Automatic selection Metadata Pattern recognition Data mining Open source software Recommendation Open systems Learning algorithm
Online Access	Get full text

Cover

Loading…

Abstract	Choosing a suitable classifier for a given dataset is an important part of developing a pattern recognition system. Since a large variety of classification algorithms are proposed in literature, non-experts do not know which method should be used in order to obtain good classification results on their data. Meta-learning tries to address this problem by recommending promising classifiers based on meta-features computed from a given dataset. In this paper, we empirically evaluate five different categories of state-of-the-art meta-features for their suitability in predicting classification accuracies of several widely used classifiers (including Support Vector Machines, Neural Networks, Random Forests, Decision Trees, and Logistic Regression). Based on the evaluation results, we have developed the first open source meta-learning system that is capable of accurately predicting accuracies of target classifiers. The user provides a dataset as input and gets an automatically created high-performance ready-to-use pattern recognition system in a few simple steps. A user study of the system with non-experts showed that the users were able to develop more accurate pattern recognition systems in significantly less development time when using our system as compared to using a state-of-the-art data mining software.
AbstractList	Choosing a suitable classifier for a given dataset is an important part of developing a pattern recognition system. Since a large variety of classification algorithms are proposed in literature, non-experts do not know which method should be used in order to obtain good classification results on their data. Meta-learning tries to address this problem by recommending promising classifiers based on meta-features computed from a given dataset. In this paper, we empirically evaluate five different categories of state-of-the-art meta-features for their suitability in predicting classification accuracies of several widely used classifiers (including Support Vector Machines, Neural Networks, Random Forests, Decision Trees, and Logistic Regression). Based on the evaluation results, we have developed the first open source meta-learning system that is capable of accurately predicting accuracies of target classifiers. The user provides a dataset as input and gets an automatically created high-performance ready-to-use pattern recognition system in a few simple steps. A user study of the system with non-experts showed that the users were able to develop more accurate pattern recognition systems in significantly less development time when using our system as compared to using a state-of-the-art data mining software.
Author	Goldstein, Markus Reif, Matthias Dengel, Andreas Breuel, Thomas Shafait, Faisal
Author_xml	– sequence: 1 givenname: Matthias surname: Reif fullname: Reif, Matthias email: matthias.reif@dfki.de organization: German Research Center for Artificial Intelligence (DFKI) – sequence: 2 givenname: Faisal surname: Shafait fullname: Shafait, Faisal organization: German Research Center for Artificial Intelligence (DFKI) – sequence: 3 givenname: Markus surname: Goldstein fullname: Goldstein, Markus organization: German Research Center for Artificial Intelligence (DFKI) – sequence: 4 givenname: Thomas surname: Breuel fullname: Breuel, Thomas organization: Department of Computer Science, University of Kaiserslautern – sequence: 5 givenname: Andreas surname: Dengel fullname: Dengel, Andreas organization: German Research Center for Artificial Intelligence (DFKI)
BackLink	http://pascal-francis.inist.fr/vibad/index.php?action=getRecordDetail&idt=28188813$$DView record in Pascal Francis
BookMark	eNp9kE1LAzEQhoNUsK3-AG978ZiayexH9liKWqHgRcFbyIaJpGx3S7IF7a83ZaUHD73MzOF9ZphnxiZd3xFj9yAWIET1GFPNcy5AciGV4McrNoUckVdF8Tk5zzncsFmMWyEQUaopWywPQ78zg7eZbU2M3nkKWaSW7OD7LnN9yNIpTt97CkO8ZdfOtJHu_vqcfTw_va_WfPP28rpabrhFUAPPSUokqkoLhK5WytWuhqIBKKnCojSQU2mkwkYWspYlYmVtI2vXCANYGpyzh3Hv3kRrWhdMZ33U--B3JvxoqUApBZhyMOZs6GMM5M4REPokRo9idBKjT2L0MTHVP8b6wZy-HYLx7UVSjmRMV7ovCnrbH0KXRFyAfgGEVHj8
CitedBy_id	crossref_primary_10_3233_IDA_205494 crossref_primary_10_1016_j_knosys_2017_06_024 crossref_primary_10_1007_s10994_023_06359_0 crossref_primary_10_1016_j_asoc_2022_109969 crossref_primary_10_1145_3418228 crossref_primary_10_1109_ACCESS_2023_3273530 crossref_primary_10_1016_j_eswa_2021_115589 crossref_primary_10_1186_s40537_023_00687_7 crossref_primary_10_1016_j_softx_2021_100919 crossref_primary_10_1007_s10044_015_0503_1 crossref_primary_10_1016_j_eswa_2021_114694 crossref_primary_10_1007_s10044_016_0583_6 crossref_primary_10_1109_TNNLS_2021_3130896 crossref_primary_10_1007_s10618_024_01002_5 crossref_primary_10_1007_s10462_013_9406_y crossref_primary_10_1109_ACCESS_2024_3365586 crossref_primary_10_1007_s13042_016_0571_6 crossref_primary_10_1007_s12599_021_00708_w crossref_primary_10_1108_JM2_03_2021_0065 crossref_primary_10_1109_ACCESS_2020_2964726 crossref_primary_10_1016_j_neucom_2016_08_013 crossref_primary_10_1016_j_knosys_2018_07_015 crossref_primary_10_1016_j_ifacol_2021_08_174 crossref_primary_10_1016_j_compbiomed_2021_104903 crossref_primary_10_1016_j_knosys_2021_108101 crossref_primary_10_3233_JIFS_169141 crossref_primary_10_1109_ACCESS_2021_3099689 crossref_primary_10_3233_IDA_194803 crossref_primary_10_1016_j_eswa_2016_11_034 crossref_primary_10_1109_ACCESS_2020_2980798 crossref_primary_10_1007_s10994_017_5683_z crossref_primary_10_1145_3360601 crossref_primary_10_3233_JIFS_161062 crossref_primary_10_1007_s10994_017_5629_5 crossref_primary_10_1007_s10845_024_02479_z crossref_primary_10_1186_s13677_023_00542_3 crossref_primary_10_1016_j_knosys_2021_106933 crossref_primary_10_32604_cmes_2025_058566 crossref_primary_10_1016_j_patcog_2022_108671 crossref_primary_10_1109_ACCESS_2024_3524908 crossref_primary_10_3389_fonc_2022_893424 crossref_primary_10_1016_j_datak_2019_101727 crossref_primary_10_1016_j_ssci_2021_105315 crossref_primary_10_1016_j_ins_2014_12_044 crossref_primary_10_1016_j_iot_2021_100432 crossref_primary_10_1007_s00170_022_08761_9 crossref_primary_10_1016_j_eswa_2017_01_013 crossref_primary_10_4018_IJERTCS_2019100102 crossref_primary_10_1016_j_tbench_2024_100150 crossref_primary_10_1371_journal_pone_0312768 crossref_primary_10_1186_s40537_022_00612_4 crossref_primary_10_1007_s10618_021_00737_9 crossref_primary_10_1016_j_ins_2019_06_005 crossref_primary_10_1016_j_cl_2018_07_002 crossref_primary_10_1109_TNNLS_2024_3366615 crossref_primary_10_1016_j_eswa_2023_119944 crossref_primary_10_1016_j_jpse_2024_100244 crossref_primary_10_1016_j_patcog_2013_10_009
Cites_doi	10.1007/978-3-540-48247-5_52 10.1016/j.patrec.2011.04.010 10.1007/3-540-44795-4_3 10.1162/neco.1996.8.7.1341 10.1145/1150402.1150531 10.1109/ICMLA.2005.65 10.1080/08839519508945477 10.1016/B978-1-55860-335-6.50023-4 10.1007/3-540-45357-1_26 10.1023/A:1021713901879 10.1007/978-3-540-87656-4_57 10.1007/3-540-45372-5_32 10.1007/3-540-36182-0_14 10.1007/3-540-60428-6_16 10.1109/34.809107 10.1016/S0065-2458(08)60520-3 10.1007/978-3-642-20980-2_9 10.1016/j.asoc.2004.12.002 10.1080/01621459.1989.10478752 10.1023/A:1010933404324
ContentType	Journal Article
Copyright	Springer-Verlag London Limited 2012 2015 INIST-CNRS
Copyright_xml	– notice: Springer-Verlag London Limited 2012 – notice: 2015 INIST-CNRS
DBID	AAYXX CITATION IQODW
DOI	10.1007/s10044-012-0280-z
DatabaseName	CrossRef Pascal-Francis
DatabaseTitle	CrossRef
DatabaseTitleList
DeliveryMethod	fulltext_linktorsrc
Discipline	Applied Sciences Computer Science
EISSN	1433-755X
EndPage	96
ExternalDocumentID	28188813 10_1007_s10044_012_0280_z
GroupedDBID	-59 -5G -BR -EM -Y2 -~C .86 .DC .VR 06D 0R~ 0VY 123 1N0 1SB 203 29O 2J2 2JN 2JY 2KG 2LR 2P1 2VQ 2~H 30V 4.4 406 408 409 40D 40E 5VS 67Z 6NX 8TC 8UJ 95- 95. 95~ 96X AAAVM AABHQ AACDK AAHNG AAIAL AAJBT AAJKR AANZL AARHV AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYQN AAYTO AAYZH ABAKF ABBBX ABBXA ABDZT ABECU ABFTD ABFTV ABHLI ABHQN ABJNI ABJOX ABKCH ABKTR ABMNI ABMQK ABNWP ABQBU ABQSL ABSXP ABTEG ABTHY ABTKH ABTMW ABULA ABWNU ABXPI ACAOD ACBXY ACDTI ACGFO ACGFS ACHSB ACHXU ACKNC ACMDZ ACMLO ACOKC ACOMO ACPIV ACREN ACSNA ACZOJ ADHHG ADHIR ADINQ ADKNI ADKPE ADRFC ADTPH ADURQ ADYFF ADYOE ADZKW AEBTG AEFQL AEGAL AEGNC AEJHL AEJRE AEKMD AEMSY AENEX AEOHA AEPYU AESKC AETLH AEVLU AEXYK AFBBN AFGCZ AFLOW AFQWF AFWTZ AFYQB AFZKB AGAYW AGDGC AGGDS AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHKAY AHSBF AHYZX AIAKS AIGIU AIIXL AILAN AITGF AJBLW AJRNO AJZVZ ALMA_UNASSIGNED_HOLDINGS ALWAN AMKLP AMTXH AMXSW AMYLF AMYQR AOCGG ARMRJ ASPBG AVWKF AXYYD AYJHY AZFZN B-. BA0 BDATZ BGNMA BSONS CAG COF CSCUP DDRTE DL5 DNIVK DPUIP DU5 EBLON EBS EIOEI EJD ESBYG F5P FEDTE FERAY FFXSO FIGPU FINBP FNLPD FRRFC FSGXE FWDCC GGCAI GGRSB GJIRD GNWQR GQ6 GQ7 GQ8 GXS H13 HF~ HG5 HG6 HMJXF HQYDN HRMNR HVGLF HZ~ I09 IHE IJ- IKXTQ IWAJR IXC IXD IXE IZIGR IZQ I~X I~Z J-C J0Z J9A JBSCW JCJTX JZLTJ KDC KOV LAS LLZTM M4Y MA- N2Q N9A NB0 NPVJJ NQJWS NU0 O9- O93 O9J OAM P2P P9O PF0 PT4 PT5 QOS R89 R9I RIG RNI ROL RPX RSV RZK S16 S1Z S27 S3B SAP SCO SDH SHX SISQX SJYHP SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW STPWE SZN T13 TSG TSK TSV TUC U2A UG4 UOJIU UTJUX UZXMN VC2 VFIZW W23 W48 WK8 YLTOR Z45 Z7R Z7X Z81 Z83 Z88 ZMTXR ~A9 AAPKM AAYXX ABBRH ABDBE ABFSG ACSTC ADHKG ADKFA AEZWR AFDZB AFHIU AFOHR AGQPQ AHPBZ AHWEU AIXLP ATHPR AYFIA CITATION IQODW
ID	FETCH-LOGICAL-c318t-4e223ee76c1e3f988f9f915b116e7356a14e6a283b252926337ccb29fb0a136a3
IEDL.DBID	U2A
ISSN	1433-7541
IngestDate	Wed Apr 02 07:27:33 EDT 2025 Tue Jul 01 01:15:13 EDT 2025 Thu Apr 24 22:59:38 EDT 2025 Fri Feb 21 02:29:20 EST 2025
IsPeerReviewed	true
IsScholarly	true
Issue	1
Keywords	Regression Classifier recommendation Classifier selection Meta-features Meta-learning Landmarking High performance Data analysis Automatic classification Automatic selection Metadata Pattern recognition Data mining Open source software Recommendation Open systems Learning algorithm
Language	English
License	http://www.springer.com/tdm CC BY 4.0
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c318t-4e223ee76c1e3f988f9f915b116e7356a14e6a283b252926337ccb29fb0a136a3
PageCount	14
ParticipantIDs	pascalfrancis_primary_28188813 crossref_primary_10_1007_s10044_012_0280_z crossref_citationtrail_10_1007_s10044_012_0280_z springer_journals_10_1007_s10044_012_0280_z
ProviderPackageCode	CITATION AAYXX
PublicationCentury	2000
PublicationDate	2014-02-01
PublicationDateYYYYMMDD	2014-02-01
PublicationDate_xml	– month: 02 year: 2014 text: 2014-02-01 day: 01
PublicationDecade	2010
PublicationPlace	London
PublicationPlace_xml	– name: London – name: Heidelberg
PublicationTitle	Pattern analysis and applications : PAA
PublicationTitleAbbrev	Pattern Anal Applic
PublicationYear	2014
Publisher	Springer London Springer
Publisher_xml	– name: Springer London – name: Springer
References	Quinlan R (2002) Data mining tools see5 and c5.0. http://www.rulequest.com/see5-info.html Chang CC, Lin CJ LIBSVM: a library for support vector machines (2001). Software available at http://www.csie.ntu.edu.tw/∼cjlin/libsvm BreimanLRandom forestsMach Learn20014553210.1023/A:10109334043241007.68152 Asuncion A, Newman D UCI machine learning repository (2007) http://www.ics.uci.edu/~mlearn/MLRepository.html University of California, Irvine, School of Information and Computer Sciences Mierswa I, Wurst M, Klinkenberg R, Scholz M, Euler T (2006) Yale: Rapid prototyping for complex data mining tasks. In: Ungar L, Craven M, Gunopulos D, Eliassi-Rad T (eds.) KDD ’06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, New York, NY, USA, pp 935–940 Piatetsky-Shapiro G (2010) Data mining / analytic tools used poll http://www.kdnuggets.com/polls/2010/data-mining-analytics-tools.html Giraud-Carrier C (2005) The data mining advisor: meta-learning at the service of practitioners. In: Proceedings of the fourth international conference on machine learning and applications, 2005, pp 113–119 KingRDFengCSutherlandAStatlog: comparison of classification algorithms on large real-world problemsAppl Artif Intell19959328933310.1080/08839519508945477 Quinlan JR (1992) Learning with continuous classes. In Proceedings AI’92, pp. 343–348 QiaoZZhouLHuangJZSparse linear discriminant analysis with applications to high dimensional low sample size dataIAENG Int J Appl Math200939148601229.620862493149 Todorovski L, Brazdil P, Soares C (2000) Report on the experiments with feature selection in meta-level learning. In: Proceedings of the PKDD-00 Workshop on Data Mining, Decision Support, Meta-Learning and ILP, pp 27–39 Hilario M, Nguyen P, Do H, Woznica A, Kalousis A (2011) Ontology-based meta-mining of knowledge discovery workflows. In: Jankowski N, Duch W, Grąbczewski K (eds.) Meta-Learning in Computational Intelligence, Studies in Computational Intelligence, vol. 358, Springer Heidelberg, pp 273–315 Michie D, Spiegelhalter D, Taylor C (1994) Machine Learning, Neural & Statistical Classification. Ellis Horwood, Chichester RendellLChoHEmpirical learning as a function of concept characterMach Learn19905267298 WolpertDHThe lack of a priori distinctions between learning algorithmsNeural Comput.1996871341139010.1162/neco.1996.8.7.1341 Bensusan H, Giraud-Carrier C (2000) Casa batló is in passeig de gràcia or how landmark performances can describe tasks. In: Proceedings of the ECML-00 workshop on meta-learning: building automatic advice strategies for model selection and method combination, pp. 29–46 Bensusan H, Giraud-Carrier CG (2000) Discovering task neighbourhoods through landmark learning performances. In: PKDD ’00: Proceedings of the 4th European Conference on Principles of Data Mining and Knowledge Discovery, Springer-Verlag, London, UK, pp 325–330 Kietz JU, Serban F, Bernstein A, Fischer S (2010) Data mining workflow templates for intelligent discovery assistance and auto-experimentation. In: Proceedings of the ECML/PKDD-10 Workshop on Third Generation Data Mining: Towards Service-Oriented Knowledge Discovery, pp 1–12 BrazdilPSoaresCda CostaJPRanking learning algorithms: Using IBL and meta-learning on accuracy and time resultsMach Learn200350325127710.1023/A:10217139018791033.68082 Kalousis A, Hilario M (2001) Feature selection for meta-learning. In: Cheung D, Williams G, Li Q (eds.) Advances in Knowledge Discovery and Data Mining, Lecture Notes in Computer Science, vol. 2035, Springer Heidelberg, pp 222–233 Köpf C, Taylor C, Keller J (2000) Meta-analysis: from data characterisation for meta-learning to meta-regression. In: Proceedings of the PKDD-00 Workshop on Data Mining, Decision Support, Meta-Learning and ILP Bensusan H, Giraud-Carrier C, Kennedy C (2000) A higher-order approach to meta-learning. In: Proceedings of the ECML’2000 workshop on meta-learning: building automatic advice strategies for model selection and method combination, pp. 109–117 Segrera S, Pinho J, Moreno M (2008) Information-theoretic measures for meta-learning. In: Corchado E, Abraham A, Pedrycz W (eds.) Hybrid Artificial Intelligence Systems, Lecture Notes in Computer Science, vol. 5271, Springer, Heidelberg, pp 458–465 FriedmanJHRegularized discriminant analysisJ Am Stat Assoc19898440516517510.1080/01621459.1989.10478752 Fürnkranz J, Petrak J (2001) An evaluation of landmarking variants. In: C. Giraud-Carrier, N. Lavrač, S. Moyle, B. Kavšek (eds.) Proceedings of the ECML/PKDD workshop on integrating aspects of data mining, decision support and meta-learning (IDDM-2001), Freiburg, Germany, pp 57–68 Peng Y, Flach P, Soares C, Brazdil P (2002) Improved dataset characterisation for meta-learning. In: S. Lange, K. Satoh, C. Smith (eds.) Discovery Science, Lecture Notes in Computer Science, vol. 2534, Springer, Heidelberg, pp 193–208 Bensusan H, Kalousis A (2001) Estimating the predictive accuracy of a classifier. In: De Raedt L, Flach P (eds.) Machine Learning: ECML 2001, Lecture Notes in Computer Science, vol. 2167 Springer, Berlin, pp 25–36 Gama J, Brazdil P (1995) Characterization of classification algorithms. In: C. Pinto-Ferreira, N. Mamede (eds.) Progress in artificial intelligence, Lecture Notes in Computer Science, vol. 990, Springer Heidelberg, pp 189–200 Pfahringer B, Bensusan H, Giraud-Carrier C (2000) Meta-learning by landmarking various learning algorithms. In: In Proceedings of the Seventeenth international conference on machine learning, Morgan Kaufmann, pp 743–750 Abdelmessih SD, Shafait F, Reif M, Goldstein M (2010) Landmarking for meta-learning using RapidMiner. In: RapidMiner community meeting and conference Engels R, Theusinger C (1998) Using a data metric for preprocessing advice for data mining applications. In: Proceedings of the European Conference on artificial intelligence (ECAI-98, Wiley, pp 430–434 FraschJVLodwichAShafaitFBreuelTMA bayes-true data generator for evaluation of supervised and unsupervised learning methodsPattern Recogn Lett201132111523153110.1016/j.patrec.2011.04.010 John GH, Kohavi R, Pfleger K (1994) Irrelevant features and the subset selection problem. In: International Conference on machine learning, Morgan Kaufmann, pp 121–129 SohnSYMeta analysis of classification algorithms for pattern recognitionIEEE Trans Pattern Anal Mach Intell199921111137114410.1109/34.809107 CortesCVapnikVSupport-vector networksMach Learn19952032732970831.68098 Esprit project METAL (#26.357): A meta-learning assistant for providing user support in data mining and machine learning (1999–2002). http://www.ofai.at/research/impml/metal Quinlan JR Cubist. http://www.rulequest.com/cubist-info.html Lindner G, Studer R (1999) Ast: support for algorithm selection with a cbr approach. In: Recent Advances in Meta-Learning and Future Work, pp 418–423 Vlachos P StatLib datasets archive (1998) http://lib.stat.cmu.edu Department of Statistics, Carnegie Mellon University AliSSmithKAOn learning algorithm selection for classificationApplied Soft Comput2006611913810.1016/j.asoc.2004.12.002 Brazdil PB, Soares C (2000) Zoomed ranking: Selection of classification algorithms based on relevant performance information. In: Proceedings of principles of data mining and knowledge discovery, 4th European conference (PKDD-2000). Springer, pp 126–135 RiceJRThe algorithm selection problemAdv Comput19761565118 280_CR9 JV Frasch (280_CR14) 2011; 32 JR Rice (280_CR37) 1976; 15 280_CR7 280_CR6 280_CR5 280_CR4 SY Sohn (280_CR39) 1999; 21 L Breiman (280_CR10) 2001; 45 P Brazdil (280_CR8) 2003; 50 280_CR17 280_CR18 DH Wolpert (280_CR42) 1996; 8 280_CR16 280_CR38 S Ali (280_CR2) 2006; 6 280_CR19 280_CR3 280_CR31 280_CR1 280_CR30 280_CR13 280_CR35 280_CR11 280_CR33 C Cortes (280_CR12) 1995; 20 280_CR34 RD King (280_CR23) 1995; 9 L Rendell (280_CR36) 1990; 5 280_CR28 280_CR29 280_CR26 280_CR27 280_CR20 JH Friedman (280_CR15) 1989; 84 280_CR21 280_CR40 280_CR41 280_CR24 Z Qiao (280_CR32) 2009; 39 280_CR25 280_CR22
References_xml	– reference: Bensusan H, Giraud-Carrier C, Kennedy C (2000) A higher-order approach to meta-learning. In: Proceedings of the ECML’2000 workshop on meta-learning: building automatic advice strategies for model selection and method combination, pp. 109–117 – reference: FraschJVLodwichAShafaitFBreuelTMA bayes-true data generator for evaluation of supervised and unsupervised learning methodsPattern Recogn Lett201132111523153110.1016/j.patrec.2011.04.010 – reference: Bensusan H, Giraud-Carrier C (2000) Casa batló is in passeig de gràcia or how landmark performances can describe tasks. In: Proceedings of the ECML-00 workshop on meta-learning: building automatic advice strategies for model selection and method combination, pp. 29–46 – reference: Fürnkranz J, Petrak J (2001) An evaluation of landmarking variants. In: C. Giraud-Carrier, N. Lavrač, S. Moyle, B. Kavšek (eds.) Proceedings of the ECML/PKDD workshop on integrating aspects of data mining, decision support and meta-learning (IDDM-2001), Freiburg, Germany, pp 57–68 – reference: Peng Y, Flach P, Soares C, Brazdil P (2002) Improved dataset characterisation for meta-learning. In: S. Lange, K. Satoh, C. Smith (eds.) Discovery Science, Lecture Notes in Computer Science, vol. 2534, Springer, Heidelberg, pp 193–208 – reference: Brazdil PB, Soares C (2000) Zoomed ranking: Selection of classification algorithms based on relevant performance information. In: Proceedings of principles of data mining and knowledge discovery, 4th European conference (PKDD-2000). Springer, pp 126–135 – reference: Pfahringer B, Bensusan H, Giraud-Carrier C (2000) Meta-learning by landmarking various learning algorithms. In: In Proceedings of the Seventeenth international conference on machine learning, Morgan Kaufmann, pp 743–750 – reference: Bensusan H, Kalousis A (2001) Estimating the predictive accuracy of a classifier. In: De Raedt L, Flach P (eds.) Machine Learning: ECML 2001, Lecture Notes in Computer Science, vol. 2167 Springer, Berlin, pp 25–36 – reference: Gama J, Brazdil P (1995) Characterization of classification algorithms. In: C. Pinto-Ferreira, N. Mamede (eds.) Progress in artificial intelligence, Lecture Notes in Computer Science, vol. 990, Springer Heidelberg, pp 189–200 – reference: AliSSmithKAOn learning algorithm selection for classificationApplied Soft Comput2006611913810.1016/j.asoc.2004.12.002 – reference: Engels R, Theusinger C (1998) Using a data metric for preprocessing advice for data mining applications. In: Proceedings of the European Conference on artificial intelligence (ECAI-98, Wiley, pp 430–434 – reference: Mierswa I, Wurst M, Klinkenberg R, Scholz M, Euler T (2006) Yale: Rapid prototyping for complex data mining tasks. In: Ungar L, Craven M, Gunopulos D, Eliassi-Rad T (eds.) KDD ’06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, ACM, New York, NY, USA, pp 935–940 – reference: Asuncion A, Newman D UCI machine learning repository (2007) http://www.ics.uci.edu/~mlearn/MLRepository.html University of California, Irvine, School of Information and Computer Sciences – reference: Esprit project METAL (#26.357): A meta-learning assistant for providing user support in data mining and machine learning (1999–2002). http://www.ofai.at/research/impml/metal/ – reference: Vlachos P StatLib datasets archive (1998) http://lib.stat.cmu.edu Department of Statistics, Carnegie Mellon University – reference: RiceJRThe algorithm selection problemAdv Comput19761565118 – reference: Todorovski L, Brazdil P, Soares C (2000) Report on the experiments with feature selection in meta-level learning. In: Proceedings of the PKDD-00 Workshop on Data Mining, Decision Support, Meta-Learning and ILP, pp 27–39 – reference: Bensusan H, Giraud-Carrier CG (2000) Discovering task neighbourhoods through landmark learning performances. In: PKDD ’00: Proceedings of the 4th European Conference on Principles of Data Mining and Knowledge Discovery, Springer-Verlag, London, UK, pp 325–330 – reference: Michie D, Spiegelhalter D, Taylor C (1994) Machine Learning, Neural & Statistical Classification. Ellis Horwood, Chichester – reference: Quinlan JR Cubist. http://www.rulequest.com/cubist-info.html – reference: CortesCVapnikVSupport-vector networksMach Learn19952032732970831.68098 – reference: Piatetsky-Shapiro G (2010) Data mining / analytic tools used poll http://www.kdnuggets.com/polls/2010/data-mining-analytics-tools.html – reference: Kalousis A, Hilario M (2001) Feature selection for meta-learning. In: Cheung D, Williams G, Li Q (eds.) Advances in Knowledge Discovery and Data Mining, Lecture Notes in Computer Science, vol. 2035, Springer Heidelberg, pp 222–233 – reference: Lindner G, Studer R (1999) Ast: support for algorithm selection with a cbr approach. In: Recent Advances in Meta-Learning and Future Work, pp 418–423 – reference: WolpertDHThe lack of a priori distinctions between learning algorithmsNeural Comput.1996871341139010.1162/neco.1996.8.7.1341 – reference: RendellLChoHEmpirical learning as a function of concept characterMach Learn19905267298 – reference: Hilario M, Nguyen P, Do H, Woznica A, Kalousis A (2011) Ontology-based meta-mining of knowledge discovery workflows. In: Jankowski N, Duch W, Grąbczewski K (eds.) Meta-Learning in Computational Intelligence, Studies in Computational Intelligence, vol. 358, Springer Heidelberg, pp 273–315 – reference: QiaoZZhouLHuangJZSparse linear discriminant analysis with applications to high dimensional low sample size dataIAENG Int J Appl Math200939148601229.620862493149 – reference: KingRDFengCSutherlandAStatlog: comparison of classification algorithms on large real-world problemsAppl Artif Intell19959328933310.1080/08839519508945477 – reference: Köpf C, Taylor C, Keller J (2000) Meta-analysis: from data characterisation for meta-learning to meta-regression. In: Proceedings of the PKDD-00 Workshop on Data Mining, Decision Support, Meta-Learning and ILP – reference: Quinlan JR (1992) Learning with continuous classes. In Proceedings AI’92, pp. 343–348 – reference: Giraud-Carrier C (2005) The data mining advisor: meta-learning at the service of practitioners. In: Proceedings of the fourth international conference on machine learning and applications, 2005, pp 113–119 – reference: John GH, Kohavi R, Pfleger K (1994) Irrelevant features and the subset selection problem. In: International Conference on machine learning, Morgan Kaufmann, pp 121–129 – reference: BrazdilPSoaresCda CostaJPRanking learning algorithms: Using IBL and meta-learning on accuracy and time resultsMach Learn200350325127710.1023/A:10217139018791033.68082 – reference: BreimanLRandom forestsMach Learn20014553210.1023/A:10109334043241007.68152 – reference: Chang CC, Lin CJ LIBSVM: a library for support vector machines (2001). Software available at http://www.csie.ntu.edu.tw/∼cjlin/libsvm – reference: Segrera S, Pinho J, Moreno M (2008) Information-theoretic measures for meta-learning. In: Corchado E, Abraham A, Pedrycz W (eds.) Hybrid Artificial Intelligence Systems, Lecture Notes in Computer Science, vol. 5271, Springer, Heidelberg, pp 458–465 – reference: SohnSYMeta analysis of classification algorithms for pattern recognitionIEEE Trans Pattern Anal Mach Intell199921111137114410.1109/34.809107 – reference: FriedmanJHRegularized discriminant analysisJ Am Stat Assoc19898440516517510.1080/01621459.1989.10478752 – reference: Quinlan R (2002) Data mining tools see5 and c5.0. http://www.rulequest.com/see5-info.html – reference: Abdelmessih SD, Shafait F, Reif M, Goldstein M (2010) Landmarking for meta-learning using RapidMiner. In: RapidMiner community meeting and conference – reference: Kietz JU, Serban F, Bernstein A, Fischer S (2010) Data mining workflow templates for intelligent discovery assistance and auto-experimentation. In: Proceedings of the ECML/PKDD-10 Workshop on Third Generation Data Mining: Towards Service-Oriented Knowledge Discovery, pp 1–12 – ident: 280_CR40 – volume: 20 start-page: 273 issue: 3 year: 1995 ident: 280_CR12 publication-title: Mach Learn – ident: 280_CR13 – ident: 280_CR1 – ident: 280_CR25 doi: 10.1007/978-3-540-48247-5_52 – volume: 39 start-page: 48 issue: 1 year: 2009 ident: 280_CR32 publication-title: IAENG Int J Appl Math – volume: 32 start-page: 1523 issue: 11 year: 2011 ident: 280_CR14 publication-title: Pattern Recogn Lett doi: 10.1016/j.patrec.2011.04.010 – ident: 280_CR3 – ident: 280_CR34 – ident: 280_CR7 doi: 10.1007/3-540-44795-4_3 – ident: 280_CR5 – ident: 280_CR11 – volume: 8 start-page: 1341 issue: 7 year: 1996 ident: 280_CR42 publication-title: Neural Comput. doi: 10.1162/neco.1996.8.7.1341 – ident: 280_CR28 doi: 10.1145/1150402.1150531 – ident: 280_CR18 doi: 10.1109/ICMLA.2005.65 – ident: 280_CR9 – volume: 9 start-page: 289 issue: 3 year: 1995 ident: 280_CR23 publication-title: Appl Artif Intell doi: 10.1080/08839519508945477 – ident: 280_CR20 doi: 10.1016/B978-1-55860-335-6.50023-4 – ident: 280_CR21 doi: 10.1007/3-540-45357-1_26 – ident: 280_CR31 – ident: 280_CR33 – volume: 50 start-page: 251 issue: 3 year: 2003 ident: 280_CR8 publication-title: Mach Learn doi: 10.1023/A:1021713901879 – ident: 280_CR27 – ident: 280_CR38 doi: 10.1007/978-3-540-87656-4_57 – ident: 280_CR6 doi: 10.1007/3-540-45372-5_32 – ident: 280_CR29 doi: 10.1007/3-540-36182-0_14 – ident: 280_CR16 – ident: 280_CR41 – ident: 280_CR17 doi: 10.1007/3-540-60428-6_16 – volume: 21 start-page: 1137 issue: 11 year: 1999 ident: 280_CR39 publication-title: IEEE Trans Pattern Anal Mach Intell doi: 10.1109/34.809107 – ident: 280_CR35 – ident: 280_CR4 – ident: 280_CR30 – volume: 15 start-page: 65 year: 1976 ident: 280_CR37 publication-title: Adv Comput doi: 10.1016/S0065-2458(08)60520-3 – ident: 280_CR19 doi: 10.1007/978-3-642-20980-2_9 – ident: 280_CR24 – ident: 280_CR26 – ident: 280_CR22 – volume: 6 start-page: 119 year: 2006 ident: 280_CR2 publication-title: Applied Soft Comput doi: 10.1016/j.asoc.2004.12.002 – volume: 5 start-page: 267 year: 1990 ident: 280_CR36 publication-title: Mach Learn – volume: 84 start-page: 165 issue: 405 year: 1989 ident: 280_CR15 publication-title: J Am Stat Assoc doi: 10.1080/01621459.1989.10478752 – volume: 45 start-page: 5 year: 2001 ident: 280_CR10 publication-title: Mach Learn doi: 10.1023/A:1010933404324
SSID	ssj0033328
Score	2.38161
Snippet	Choosing a suitable classifier for a given dataset is an important part of developing a pattern recognition system. Since a large variety of classification...
SourceID	pascalfrancis crossref springer
SourceType	Index Database Enrichment Source Publisher
StartPage	83
SubjectTerms	Algorithmics. Computability. Computer arithmetics Applied sciences Computer Science Computer science; control theory; systems Data processing. List processing. Character string processing Exact sciences and technology Memory organisation. Data processing Pattern Recognition Software Theoretical Advances Theoretical computing
Title	Automatic classifier selection for non-experts
URI	https://link.springer.com/article/10.1007/s10044-012-0280-z
Volume	17
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
link	http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA7aXgTxLdZH2YMnJbJJNtnNcZHWoujJQj0tSZqAIFrc7aW_3snuplp8gNeQZGGSyXyzM_MNQufKcm5dIrEyQvu_VQpLag0onraxcIIY6quR7x_EaJzcTvikreMuQ7Z7CEnWL_WXYrc48RkTFPtwIF6soy73rjtc4jHNw_PLGKsbqgIOYDjlCQmhzJ-2WDFGmzNVglxc09DiW2S0NjjDHbTVIsUob452F63Z1z203aLGqNXJEoZCY4Ywto-u8nn1VnOxRsaj42cHxi8q65Y3cA4RANUI3H5c0_tX5QEaDweP1yPcNkbABlSwwokFo25tKgyxzMksc9JJwjUhwqaMC0USKxQAB005lVQwlhqjqXQ6VoQJxQ5RB75ij1DEnNUiS4iZZoCMvP80tVakilATc8VlD8VBQoVpWcN984qX4pPv2Au1AKEWXqjFooculktmDWXGX5P7K2JfrvAsVVlGWA9dhnMoWvUqf9_u-F-zT9AG4J-kScI-RZ3qfW7PAGNUuo-6-c3T3aBf360Px-zJOg
linkProvider	Springer Nature
linkToHtml	http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA5aDwriW6yPugdPSmTz3M2xiKVq21MLvS3ZNAFBtLjbS3-9k-2mWnyA15BkYSaT-bLz-BC60lYI67jC2sjc_63SWFFrwPByG0sniaG-Grk_kN0RfxyLcV3HXYRs9xCSrG7qL8VuMfcZExT7cCCer6MNwAKpz-Ma0Xa4fhljFaEq4ACGE8FJCGX-tMWKM9qe6gLk4haEFt8io5XD6eyhnRopRu2FavfRmn09QLs1aoxqmyxgKBAzhLFDdNuelW9VL9bIeHT87MD5RUVFeQN6iACoRvDsx1V7_7I4QqPO_fCui2tiBGzABEvMLTh1axNpiGVOpalTThGREyJtwoTUhFupATjkVFBFJWOJMTlVLo81YVKzY9SAr9gTFDFnc5lyYiYpICP_fppYKxNNqImFFqqJ4iChzNRdwz15xUv22e_YCzUDoWZeqNm8ia6XS6aLlhl_TW6tiH25wnepSlPCmugm6CGrzav4fbvTf82-RJvdYb-X9R4GT2doC7AQXyRkn6NG-T6zF4A3yrxVna8PFuzKmQ
linkToPdf	http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1JSwMxFA5aQQRxF-tS5-BJiU7WmRyLWupWPFjobUjSBARpizO99NebzEyqxQW8hizwkpf3zbyX7wPgTBrGjKUCSs2V_1slocBGO8dTJuaWI439a-SnHu_26f2ADWqd0zxUu4eUZPWmwbM0jYqrydBefXn4FlNfPYGhTw3C2TJYcbcx8se6j9vhKiaElOKqDhMQmDCKQlrzpykWAtP6RObORrYSt_iWJS2DT2cLbNSoMWpX27wNlsxoB2zWCDKq_TN3TUGkIbTtgsv2tBiXvKyR9kj51bpAGOWl_I3bk8iB1mg0HsGS6r_I90C_c_ty3YW1SALUzh0LSI0L8MYkXCNDrEhTK6xATCHETUIYl4gaLh2IUJhhgTkhidYKC6tiiQiXZB803CrmAETEGsVTivQwdSjJf0sNjeGJRFjHTDLRBHGwUKZrBnEvZPGWfXIfe6NmzqiZN2o2a4Lz-ZBJRZ_xV-fWgtnnIzxjVZoi0gQXYR-y2tXy36c7_FfvU7D6fNPJHu96D0dgzcEiWtVmH4NG8T41Jw56FKpVHq8PFFHO1Q
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Automatic+classifier+selection+for+non-experts&rft.jtitle=Pattern+analysis+and+applications+%3A+PAA&rft.au=Reif%2C+Matthias&rft.au=Shafait%2C+Faisal&rft.au=Goldstein%2C+Markus&rft.au=Breuel%2C+Thomas&rft.date=2014-02-01&rft.pub=Springer+London&rft.issn=1433-7541&rft.eissn=1433-755X&rft.volume=17&rft.issue=1&rft.spage=83&rft.epage=96&rft_id=info:doi/10.1007%2Fs10044-012-0280-z&rft.externalDocID=10_1007_s10044_012_0280_z
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1433-7541&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1433-7541&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1433-7541&client=summon