Completely Lazy Learning
Local classifiers are sometimes called lazy learners because they do not train a classifier until presented with a test sample. However, such methods are generally not completely lazy because the neighborhood size k (or other locality parameter) is usually chosen by cross validation on the training...
Saved in:
Published in | IEEE transactions on knowledge and data engineering Vol. 22; no. 9; pp. 1274 - 1285 |
---|---|
Main Authors | , , , |
Format | Journal Article |
Language | English |
Published |
New York, NY
IEEE
01.09.2010
IEEE Computer Society The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Local classifiers are sometimes called lazy learners because they do not train a classifier until presented with a test sample. However, such methods are generally not completely lazy because the neighborhood size k (or other locality parameter) is usually chosen by cross validation on the training set, which can require significant preprocessing and risks overfitting. We propose a simple alternative to cross validation of the neighborhood size that requires no preprocessing: instead of committing to one neighborhood size, average the discriminants for multiple neighborhoods. We show that this forms an expected estimated posterior that minimizes the expected Bregman loss with respect to the uncertainty about the neighborhood choice. We analyze this approach for six standard and state-of-the-art local classifiers, including discriminative adaptive metric kNN (DANN), a local support vector machine (SVM-KNN), hyperplane distance nearest neighbor (HKNN), and a new local Bayesian quadratic discriminant analysis (local BDA). The empirical effectiveness of this technique versus cross validation is confirmed with experiments on seven benchmark data sets, showing that similar classification performance can be attained without any training. |
---|---|
AbstractList | Local classifiers are sometimes called lazy learners because they do not train a classifier until presented with a test sample. However, such methods are generally not completely lazy because the neighborhood size k (or other locality parameter) is usually chosen by cross validation on the training set, which can require significant preprocessing and risks overfitting. We propose a simple alternative to cross validation of the neighborhood size that requires no preprocessing: instead of committing to one neighborhood size, average the discriminants for multiple neighborhoods. We show that this forms an expected estimated posterior that minimizes the expected Bregman loss with respect to the uncertainty about the neighborhood choice. We analyze this approach for six standard and state-of-the-art local classifiers, including discriminative adaptive metric kNN (DANN), a local support vector machine (SVM-KNN), hyperplane distance nearest neighbor (HKNN), and a new local Bayesian quadratic discriminant analysis (local BDA). The empirical effectiveness of this technique versus cross validation is confirmed with experiments on seven benchmark data sets, showing that similar classification performance can be attained without any training. |
Author | Srivastava, Santosh Garcia, Eric K Feldman, Sergey Gupta, Maya R |
Author_xml | – sequence: 1 givenname: Eric K surname: Garcia fullname: Garcia, Eric K email: eric.garcia@gmail.com organization: Dept. of Electr. Eng., Univ. of Washington, Seattle, WA, USA – sequence: 2 givenname: Sergey surname: Feldman fullname: Feldman, Sergey email: sergeyfeldman@gmail.com organization: Dept. of Electr. Eng., Univ. of Washington, Seattle, WA, USA – sequence: 3 givenname: Maya R surname: Gupta fullname: Gupta, Maya R email: gupta@ee.washington.edu organization: Dept. of Electr. Eng., Univ. of Washington, Seattle, WA, USA – sequence: 4 givenname: Santosh surname: Srivastava fullname: Srivastava, Santosh email: ssrivast@fhcrc.org organization: Fred Hutchinson Cancer Res. Center, Seattle, WA, USA |
BackLink | http://pascal-francis.inist.fr/vibad/index.php?action=getRecordDetail&idt=23143980$$DView record in Pascal Francis |
BookMark | eNpdkE1LAzEQhoMo2FaPguBFEPG0NZNks8lRav3Agpd6Dml2Ilu22Zq0h_rrzdLSgzlMBuaZd2beITkNXUBCroCOAah-nH88T8eMUj2GUp-QAZSlKhhoOM05FVAILqpzMkxpSSlVlYIBuZ50q3WLG2x3tzP7mwPaGJrwfUHOvG0TXh7-Efl6mc4nb8Xs8_V98jQrXBbbFFxrSRXljiupmYWyrp2rKi7FAmuvmKsWIk-S6GtLGSpnS0TlOfNCMy8FH5GHve46dj9bTBuzapLDtrUBu20yCpTinEKZybt_5LLbxpCXM0BZlR_onir2lItdShG9WcdmZeMuQ6a3yfQ2md4mk23K_P1B1SZnWx9tcE06NjEOgut84Ijc7LkGEY_lEiQwyfgfoLJu5w |
CODEN | ITKEEH |
CitedBy_id | crossref_primary_10_1007_s10586_017_1276_7 crossref_primary_10_12720_ijsps_1_2_208_211 crossref_primary_10_1016_j_patrec_2012_04_013 crossref_primary_10_1049_hve_2019_0098 crossref_primary_10_1109_TITS_2015_2497320 crossref_primary_10_1109_TSP_2011_2105484 crossref_primary_10_1016_j_patcog_2016_03_029 crossref_primary_10_1109_TPAMI_2011_142 crossref_primary_10_1111_anzs_12325 crossref_primary_10_1016_j_sigpro_2022_108796 crossref_primary_10_1016_j_patcog_2019_107122 crossref_primary_10_1016_j_patcog_2010_10_020 crossref_primary_10_1109_TNN_2010_2087415 crossref_primary_10_1109_TSMCC_2010_2103939 crossref_primary_10_1002_widm_1289 crossref_primary_10_3390_sym13112128 crossref_primary_10_1109_TMM_2017_2700204 crossref_primary_10_3390_e25081207 crossref_primary_10_3390_a13010026 crossref_primary_10_1109_TKDE_2012_35 crossref_primary_10_1155_2022_1154325 crossref_primary_10_1016_j_eswa_2018_03_005 crossref_primary_10_1016_j_neucom_2012_11_019 crossref_primary_10_1016_j_ins_2016_08_089 |
Cites_doi | 10.1214/aos/1176343886 10.1109/TPAMI.2002.1023804 10.1111/j.1467-9868.2005.00506.x 10.1109/TIT.2008.929943 10.1109/TPAMI.2005.204 10.1109/ICPR.2000.906187 10.1109/TPAMI.2006.101 10.2307/1267351 10.1023/A:1006563312922 10.1007/978-0-387-21606-5 10.1109/TIT.1965.1053726 10.1109/34.506411 10.1016/j.patrec.2005.12.016 10.1109/CVPR.2006.301 10.1016/j.patcog.2005.06.004 10.1016/S0167-8655(97)00112-8 10.21236/ADA479632 10.1109/TIP.2008.922429 10.1109/TKDE.2008.90 10.1109/TKDE.2005.53 10.1201/9780203011232 10.1109/TKDE.2007.190700 10.1145/1961189.1961199 10.1137/1.9781611973068.36 10.2202/1544-6115.1054 10.1145/502807.502809 10.2307/2291604 10.1111/1467-9868.00338 10.1109/TIT.2005.850145 10.1007/978-3-540-45072-6_6 10.2307/2289860 10.1007/978-94-017-2053-3 |
ContentType | Journal Article |
Copyright | 2015 INIST-CNRS Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Sep 2010 |
Copyright_xml | – notice: 2015 INIST-CNRS – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Sep 2010 |
DBID | 97E RIA RIE IQODW AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D F28 FR3 |
DOI | 10.1109/TKDE.2009.159 |
DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005-present IEEE All-Society Periodicals Package (ASPP) 1998-Present IEEE Electronic Library Online Pascal-Francis CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional ANTE: Abstracts in New Technology & Engineering Engineering Research Database |
DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional Engineering Research Database ANTE: Abstracts in New Technology & Engineering |
DatabaseTitleList | Technology Research Database Technology Research Database |
Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library Online url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/ sourceTypes: Publisher |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Engineering Computer Science Applied Sciences |
EISSN | 1558-2191 |
EndPage | 1285 |
ExternalDocumentID | 2720445051 10_1109_TKDE_2009_159 23143980 5161262 |
Genre | orig-research |
GroupedDBID | -~X .DC 0R~ 1OL 29I 4.4 5GY 5VS 6IK 97E 9M8 AAJGR AASAJ AAYOK ABFSI ABQJQ ABVLG ACGFO ACIWK AENEX AETIX AI. AIBXA AKJIK ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 E.L EBS EJD F5P HZ~ H~9 ICLAB IEDLZ IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P PQQKQ RIA RIC RIE RIG RNI RNS RXW RZB TAE TAF TN5 UHB VH1 XFK ABPTK IQODW VOH AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D F28 FR3 |
ID | FETCH-LOGICAL-c347t-39960803c38692a15ddcc77364bedf82c7b48786efda02e8ca5ee8f32f492f643 |
IEDL.DBID | RIE |
ISSN | 1041-4347 |
IngestDate | Fri Aug 16 07:34:34 EDT 2024 Thu Oct 10 18:58:43 EDT 2024 Fri Aug 23 01:04:20 EDT 2024 Sun Oct 22 16:05:38 EDT 2023 Wed Jun 26 19:28:22 EDT 2024 |
IsPeerReviewed | true |
IsScholarly | true |
Issue | 9 |
Keywords | Bayes estimation Nearest neighbour Local search Lazy learning Discriminant analysis Bayesian estimation Statistical analysis Locality Cross validation local learning Adaptive method Hyperplane Vector support machine Metric quadratic discriminant analysis |
Language | English |
License | CC BY 4.0 |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-c347t-39960803c38692a15ddcc77364bedf82c7b48786efda02e8ca5ee8f32f492f643 |
Notes | ObjectType-Article-2 SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 23 |
PQID | 1027777195 |
PQPubID | 85438 |
PageCount | 12 |
ParticipantIDs | proquest_miscellaneous_818833015 pascalfrancis_primary_23143980 proquest_journals_1027777195 crossref_primary_10_1109_TKDE_2009_159 ieee_primary_5161262 |
PublicationCentury | 2000 |
PublicationDate | 2010-09-01 |
PublicationDateYYYYMMDD | 2010-09-01 |
PublicationDate_xml | – month: 09 year: 2010 text: 2010-09-01 day: 01 |
PublicationDecade | 2010 |
PublicationPlace | New York, NY |
PublicationPlace_xml | – name: New York, NY – name: New York |
PublicationTitle | IEEE transactions on knowledge and data engineering |
PublicationTitleAbbrev | TKDE |
PublicationYear | 2010 |
Publisher | IEEE IEEE Computer Society The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
Publisher_xml | – name: IEEE – name: IEEE Computer Society – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
References | geisser (bibttk201009127438) 1964; 26 bibttk201009127422 bibttk201009127424 bibttk201009127441 bibttk201009127421 bibttk201009127420 skalak (bibttk201009127426) 1996 petersen (bibttk201009127443) 2005 bibttk20100912743 bibttk201009127427 gupta (bibttk201009127431) 2009 bibttk20100912742 bibttk20100912741 bibttk201009127428 bay (bibttk201009127423) 1998 lehmann (bibttk201009127429) 1998 hastie (bibttk20100912745) 2001 ripley (bibttk201009127440) 2001 vincent (bibttk201009127415) 2001 liu (bibttk20100912748) 2006; 7 bibttk201009127412 bibttk201009127434 bibttk201009127411 bibttk201009127433 bibttk201009127414 bibttk201009127436 bibttk201009127435 bibttk201009127430 bibttk201009127410 bibttk201009127419 bibttk201009127416 bibttk201009127437 srivastava (bibttk201009127413) 2007; 8 bibttk201009127439 bibttk201009127417 speed (bibttk201009127425) 2003 bibttk20100912747 bibttk20100912746 liu (bibttk20100912749) 2003 bibttk20100912744 gupta (bibttk201009127432) 2006 sibson (bibttk201009127418) 1981 chang (bibttk201009127442) 2001 |
References_xml | – ident: bibttk201009127433 doi: 10.1214/aos/1176343886 – ident: bibttk20100912744 doi: 10.1109/TPAMI.2002.1023804 – ident: bibttk201009127424 doi: 10.1111/j.1467-9868.2005.00506.x – ident: bibttk201009127414 doi: 10.1109/TIT.2008.929943 – ident: bibttk201009127420 doi: 10.1109/TPAMI.2005.204 – ident: bibttk201009127435 doi: 10.1109/ICPR.2000.906187 – ident: bibttk20100912743 doi: 10.1109/TPAMI.2006.101 – ident: bibttk201009127434 doi: 10.2307/1267351 – ident: bibttk201009127427 doi: 10.1023/A:1006563312922 – start-page: 985 year: 2001 ident: bibttk201009127415 publication-title: Advances in neural information processing systems contributor: fullname: vincent – year: 2001 ident: bibttk20100912745 publication-title: The Elements of Statistical Learning doi: 10.1007/978-0-387-21606-5 contributor: fullname: hastie – year: 1996 ident: bibttk201009127426 publication-title: "Prototype Selection for Composite Nearest Neighbor Classification " contributor: fullname: skalak – year: 2005 ident: bibttk201009127443 article-title: The Matrix Cookbook contributor: fullname: petersen – ident: bibttk201009127439 doi: 10.1109/TIT.1965.1053726 – ident: bibttk201009127416 doi: 10.1109/34.506411 – start-page: 1 year: 2009 ident: bibttk201009127431 article-title: Weighted Nearest Neighbor Classifiers and First-Order Error publication-title: Proc Int'l Conf Frontiers of Interface between Statistics and Science contributor: fullname: gupta – ident: bibttk201009127436 doi: 10.1016/j.patrec.2005.12.016 – ident: bibttk20100912742 doi: 10.1109/CVPR.2006.301 – ident: bibttk201009127428 doi: 10.1016/j.patcog.2005.06.004 – ident: bibttk201009127417 doi: 10.1016/S0167-8655(97)00112-8 – year: 2006 ident: bibttk201009127432 article-title: Minimum Expected Risk Estimation for Near-Neighbor Classification doi: 10.21236/ADA479632 contributor: fullname: gupta – ident: bibttk201009127419 doi: 10.1109/TIP.2008.922429 – ident: bibttk201009127410 doi: 10.1109/TKDE.2008.90 – start-page: 37 year: 1998 ident: bibttk201009127423 article-title: Combining Nearest Neighbor Classifiers through Multiple Feature Subsets publication-title: Proc Int'l Conf Machine Learning (ICML) contributor: fullname: bay – ident: bibttk20100912747 doi: 10.1109/TKDE.2005.53 – year: 2003 ident: bibttk201009127425 publication-title: Statistical Analysis of Gene Expression Microarray Data doi: 10.1201/9780203011232 contributor: fullname: speed – start-page: 21 year: 1981 ident: bibttk201009127418 publication-title: Interpreting Multivariate Data contributor: fullname: sibson – volume: 26 start-page: 69 year: 1964 ident: bibttk201009127438 article-title: Posterior Odds for Multivariate Normal Distributions publication-title: J Royal Soc Series B Methodological contributor: fullname: geisser – ident: bibttk201009127412 doi: 10.1109/TKDE.2007.190700 – year: 1998 ident: bibttk201009127429 publication-title: Theory of Point Estimation contributor: fullname: lehmann – year: 2001 ident: bibttk201009127442 article-title: LIBSVM: A Library for Support Vector Machines doi: 10.1145/1961189.1961199 contributor: fullname: chang – volume: 7 start-page: 1135 year: 2006 ident: bibttk20100912748 article-title: New Algorithms for Efficient High-Dimensional Nonparametric Classification publication-title: J Machine Learning Research contributor: fullname: liu – ident: bibttk201009127411 doi: 10.1137/1.9781611973068.36 – year: 2001 ident: bibttk201009127440 publication-title: Pattern recognition and neural nets contributor: fullname: ripley – ident: bibttk201009127421 doi: 10.2202/1544-6115.1054 – ident: bibttk20100912746 doi: 10.1145/502807.502809 – volume: 8 start-page: 1287 year: 2007 ident: bibttk201009127413 article-title: Bayesian Quadratic Discriminant Analysis publication-title: J Machine Learning Research contributor: fullname: srivastava – ident: bibttk201009127441 doi: 10.2307/2291604 – ident: bibttk201009127422 doi: 10.1111/1467-9868.00338 – ident: bibttk201009127430 doi: 10.1109/TIT.2005.850145 – start-page: 83 year: 2003 ident: bibttk20100912749 publication-title: Lecture Notes on Computer Science doi: 10.1007/978-3-540-45072-6_6 contributor: fullname: liu – ident: bibttk201009127437 doi: 10.2307/2289860 – ident: bibttk20100912741 doi: 10.1007/978-94-017-2053-3 |
SSID | ssj0008781 |
Score | 2.2737646 |
Snippet | Local classifiers are sometimes called lazy learners because they do not train a classifier until presented with a test sample. However, such methods are... |
SourceID | proquest crossref pascalfrancis ieee |
SourceType | Aggregation Database Index Database Publisher |
StartPage | 1274 |
SubjectTerms | Applied sciences Artificial intelligence Bayesian estimation Bayesian methods Benchmark testing Classification Classifiers Computer science; control theory; systems Costs cross validation Data processing. List processing. Character string processing Discriminant analysis Error analysis Exact sciences and technology Feedback Lazy learning Learning Learning and adaptive systems local learning Memory organisation. Data processing Nearest neighbor searches Plant populations Preprocessing quadratic discriminant analysis Software Studies Support vector machine classification Support vector machines Training Training data Trains Uncertainty |
Title | Completely Lazy Learning |
URI | https://ieeexplore.ieee.org/document/5161262 https://www.proquest.com/docview/1027777195 https://search.proquest.com/docview/818833015 |
Volume | 22 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV07T8MwED6VTjBQaEENFJQBMZE2sZ3EGRG0qoAytVK3yLGdBZQimg7013POi_IYSKQoki05ubPP39nn7wCu0ENOERcETkAZc5gKEych0ndcjX6YFjhnK7M0MHsOpgv2sPSXLbhpzsJorYvgMz00r8VevlrJjVkqG_kIT4gxuHvcJeVZrcbq8rBISIreBfpElIVffJqj-eP9uGSm9Awl6c78UyRUMeGQYo0SSctUFr-scjHVTDowqz-yjDB5GW7yZCi3P_gb__sXR3BYYU77tuwkx9DSWRc6dT4HuxreXTjYISfsQd9UQK3q1w_7SWzxUa2inMBiMp7fTZ0qj4IjURC5Qw0DC3eppDyIiPB8paQMQxqwRKuUExkm6LbwQKdKuERzKXyteUpJyiKSImQ5hXa2ynQfbDeQWDGUikUJY4px6SWiuBmjhCoLrmvpxm8lXUZcuBluFBs1mKyXUYxqsKBnJNNUqoRiweU3XTTlCEMROnHXgkGtnLgabWtsgYR4eZFvgd0U4zgxmx8i06vNOkZgwilaM__s75bPYb8MDDDhYwNo5-8bfYF4I08ui472Cf-t0Ec |
link.rule.ids | 315,786,790,802,27955,27956,55107 |
linkProvider | IEEE |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LT8MwDLYmOAAHBhtohTF6QJzo1iVpkx4RbBrscdqk3ao2SS-gDrHtwH49TtuV8TjQSlWlREprJ_Znx7EBbtBCThAX-I5PGXOY4rETE-k5rkY7TEeos5VxDYwn_mDGnufevAJ35VkYrXUWfKbb5jXby1cLuTauso6H8IQYgbuPet7l-WmtUu4KnpUkRfsCrSLK-FdGzc50-NjLc1N2TVLSHQ2UlVQxAZHREmmS5MUsfsnlTNn0qzDefmYeY_LSXq_ittz8yOD43_84geMCddr3-TQ5hYpOa1DdVnSwiwVeg6Od9IR1aJgOyFf9-mGPog0-Cj_KGcz6venDwCkqKTgSCbFyqMnBIlwqqfADEnU9paTknPos1ioRRPIYDRfh60RFLtFCRp7WIqEkYQFJELScw166SHUDbNeX2JFLxYKYMcWE7MZRdjNGCVUW3G6pG77lCTPCzNBwg9CwwdS9DEJkgwV1Q5myU0EUC1rfeFG2IxBF8CRcC5pb5oTFelviCITj1Q08C-yyGVeK2f6IUr1YL0OEJoKiPPMu_h75Gg4G0_EoHD1NhpdwmIcJmGCyJuyt3tf6CtHHKm5lk-4TJCTTmw |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Completely+Lazy+Learning&rft.jtitle=IEEE+transactions+on+knowledge+and+data+engineering&rft.au=Garcia%2C+Eric+K&rft.au=Feldman%2C+Sergey&rft.au=Gupta%2C+Maya+R&rft.au=Srivastava%2C+Santosh&rft.date=2010-09-01&rft.issn=1041-4347&rft.volume=22&rft.issue=9&rft.spage=1274&rft.epage=1285&rft_id=info:doi/10.1109%2FTKDE.2009.159&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TKDE_2009_159 |
thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1041-4347&client=summon |
thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1041-4347&client=summon |
thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1041-4347&client=summon |