Calibration after bootstrap for accurate uncertainty quantification in regression models
Abstract Obtaining accurate estimates of machine learning model uncertainties on newly predicted data is essential for understanding the accuracy of the model and whether its predictions can be trusted. A common approach to such uncertainty quantification is to estimate the variance from an ensemble...
Saved in:
Published in | npj computational materials Vol. 8; no. 1; pp. 1 - 9 |
---|---|
Main Authors | , , , , , , , , , |
Format | Journal Article |
Language | English |
Published |
London
Nature Publishing Group
20.05.2022
Nature Portfolio |
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Abstract Obtaining accurate estimates of machine learning model uncertainties on newly predicted data is essential for understanding the accuracy of the model and whether its predictions can be trusted. A common approach to such uncertainty quantification is to estimate the variance from an ensemble of models, which are often generated by the generally applicable bootstrap method. In this work, we demonstrate that the direct bootstrap ensemble standard deviation is not an accurate estimate of uncertainty but that it can be simply calibrated to dramatically improve its accuracy. We demonstrate the effectiveness of this calibration method for both synthetic data and numerous physical datasets from the field of Materials Science and Engineering. The approach is motivated by applications in physical and biological science but is quite general and should be applicable for uncertainty quantification in a wide range of machine learning regression models. |
---|---|
AbstractList | Abstract Obtaining accurate estimates of machine learning model uncertainties on newly predicted data is essential for understanding the accuracy of the model and whether its predictions can be trusted. A common approach to such uncertainty quantification is to estimate the variance from an ensemble of models, which are often generated by the generally applicable bootstrap method. In this work, we demonstrate that the direct bootstrap ensemble standard deviation is not an accurate estimate of uncertainty but that it can be simply calibrated to dramatically improve its accuracy. We demonstrate the effectiveness of this calibration method for both synthetic data and numerous physical datasets from the field of Materials Science and Engineering. The approach is motivated by applications in physical and biological science but is quite general and should be applicable for uncertainty quantification in a wide range of machine learning regression models. Obtaining accurate estimates of machine learning model uncertainties on newly predicted data is essential for understanding the accuracy of the model and whether its predictions can be trusted. A common approach to such uncertainty quantification is to estimate the variance from an ensemble of models, which are often generated by the generally applicable bootstrap method. In this work, we demonstrate that the direct bootstrap ensemble standard deviation is not an accurate estimate of uncertainty but that it can be simply calibrated to dramatically improve its accuracy. We demonstrate the effectiveness of this calibration method for both synthetic data and numerous physical datasets from the field of Materials Science and Engineering. The approach is motivated by applications in physical and biological science but is quite general and should be applicable for uncertainty quantification in a wide range of machine learning regression models. |
ArticleNumber | 115 |
Author | Politowicz, Alexander Morgan, Dane Gautam, Anupraas Yang, Xiyu Li, Zhelong Du, Siqi Emory, Joshua Paul Palmer, Glenn Gupta, Grishma Jacobs, Ryan |
Author_xml | – sequence: 1 givenname: Glenn surname: Palmer fullname: Palmer, Glenn – sequence: 2 givenname: Siqi surname: Du fullname: Du, Siqi – sequence: 3 givenname: Alexander orcidid: 0000-0001-6096-2031 surname: Politowicz fullname: Politowicz, Alexander – sequence: 4 givenname: Joshua Paul surname: Emory fullname: Emory, Joshua Paul – sequence: 5 givenname: Xiyu surname: Yang fullname: Yang, Xiyu – sequence: 6 givenname: Anupraas surname: Gautam fullname: Gautam, Anupraas – sequence: 7 givenname: Grishma surname: Gupta fullname: Gupta, Grishma – sequence: 8 givenname: Zhelong surname: Li fullname: Li, Zhelong – sequence: 9 givenname: Ryan orcidid: 0000-0003-2229-6730 surname: Jacobs fullname: Jacobs, Ryan – sequence: 10 givenname: Dane orcidid: 0000-0002-4911-0046 surname: Morgan fullname: Morgan, Dane |
BookMark | eNpNUctKBDEQDKLga3_A04Dn0c5zkqMsvkDwouAt9GSSJcs6WZPMwb93dEQ8dVd3dXVBnZLDMY2ekAsKVxS4vi6CSiZaYKwF6Ixo9QE5YSC7lhsFh__6Y7IqZQsA1DDNBJyQtzXuYp-xxjQ2GKrPTZ9SLTXjvgkpN-jcNK99M43O54pxrJ_Nx4RjjSG65S6OTfab7Ev5Ru9p8LtyTo4C7opf_dYz8np3-7J-aJ-e7x_XN0-t41rWtpODRtVThT0YhR3vpeeD4MbMsDcscAPYDa5TclBuABm6wQipQ-9MzyXnZ-Rx0R0Sbu0-x3fMnzZhtD-DlDcWc41u563TIWjNtaJUCM2pQRRecZAajHeCzlqXi9Y-p4_Jl2q3acrjbN8ypTqYPVE2s9jCcjmVkn34-0rBfgdil0DsHIj9CcRq_gXysX-v |
CitedBy_id | crossref_primary_10_1016_j_energy_2024_131146 crossref_primary_10_1038_s43588_022_00280_7 crossref_primary_10_1016_j_ensm_2023_102883 crossref_primary_10_1063_5_0174943 crossref_primary_10_1016_j_coal_2023_104419 crossref_primary_10_1088_2632_2153_acee42 crossref_primary_10_1038_s41598_023_43024_x crossref_primary_10_1063_5_0136574 crossref_primary_10_1186_s13321_023_00709_9 crossref_primary_10_1016_j_matdes_2023_112491 crossref_primary_10_1088_2632_2153_ad4231 crossref_primary_10_1002_minf_202400018 crossref_primary_10_1088_2632_2153_aca7b1 crossref_primary_10_1088_1361_651X_ad4d0d crossref_primary_10_1088_2632_2153_acb9d5 crossref_primary_10_1002_aenm_202303684 crossref_primary_10_1039_D2SC04056E |
Cites_doi | 10.1023/A:1010933404324 10.1038/s41467-021-25342-8 10.1038/s41592-019-0686-2 10.1038/s41524-018-0085-8 10.1039/C9SC02298H 10.1146/annurev-matsci-070218-010015 10.1038/nn.3331 10.1002/adma.201702884 10.1016/j.commatsci.2019.06.010 10.1080/00031305.1983.10483087 10.1021/acs.jcim.0c00502 10.1002/9781119148739.ch4 10.1016/j.csda.2008.08.007 10.1093/bioinformatics/bti499 10.1038/sdata.2015.9 10.1111/j.2517-6161.1992.tb01866.x 10.1088/2632-2153/ac3eb3 10.1016/j.commatsci.2020.109544 10.1021/acs.jctc.8b00959 10.1016/j.actamat.2019.03.010 10.1007/s40192-017-0098-z 10.1016/j.commatsci.2018.04.033 10.1088/2632-2153/ab7e1a 10.1007/3-540-45014-9_1 10.1038/srep19375 10.1021/acs.jcim.8b00597 10.1038/sdata.2015.53 10.1063/5.0012405 10.1023/A:1008306431147 10.1007/978-1-4614-7138-7 |
ContentType | Journal Article |
Copyright | The Author(s) 2022. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
Copyright_xml | – notice: The Author(s) 2022. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
DBID | AAYXX CITATION 3V. 7X7 7XB 8FE 8FG 8FH 8FI 8FJ 8FK ABJCF ABUWG AFKRA AZQEC BBNVY BENPR BGLVJ BHPHI CCPQU D1I DWQXO FYUFA GHDGH GNUQQ HCIFZ K9. KB. LK8 M0S M7P PDBOC PIMPY PQEST PQQKQ PQUKI PRINS DOA |
DOI | 10.1038/s41524-022-00794-8 |
DatabaseName | CrossRef ProQuest Central (Corporate) Health & Medical Collection ProQuest Central (purchase pre-March 2016) ProQuest SciTech Collection ProQuest Technology Collection ProQuest Natural Science Collection Hospital Premium Collection Hospital Premium Collection (Alumni Edition) ProQuest Central (Alumni) (purchase pre-March 2016) Materials Science & Engineering Collection ProQuest Central (Alumni Edition) ProQuest Central ProQuest Central Essentials Biological Science Collection ProQuest Central Technology Collection ProQuest Natural Science Collection ProQuest One Community College ProQuest Materials Science Collection ProQuest Central Health Research Premium Collection Health Research Premium Collection (Alumni) ProQuest Central Student SciTech Premium Collection ProQuest Health & Medical Complete (Alumni) Materials Science Database Biological Sciences Health & Medical Collection (Alumni Edition) Biological Science Database Materials Science Collection Publicly Available Content Database ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic ProQuest One Academic UKI Edition ProQuest Central China DOAJ Directory of Open Access Journals |
DatabaseTitle | CrossRef Publicly Available Content Database ProQuest Central Student Technology Collection ProQuest Central Essentials Materials Science Collection ProQuest Health & Medical Complete (Alumni) ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Natural Science Collection ProQuest Central China ProQuest Central Health Research Premium Collection Health and Medicine Complete (Alumni Edition) Natural Science Collection ProQuest Central Korea Biological Science Collection Materials Science Database ProQuest Materials Science Collection ProQuest Biological Science Collection ProQuest One Academic Eastern Edition ProQuest Hospital Collection ProQuest Technology Collection Health Research Premium Collection (Alumni) Biological Science Database ProQuest SciTech Collection ProQuest Hospital Collection (Alumni) ProQuest Health & Medical Complete ProQuest One Academic UKI Edition Materials Science & Engineering Collection ProQuest One Academic ProQuest Central (Alumni) |
DatabaseTitleList | CrossRef Publicly Available Content Database |
Database_xml | – sequence: 1 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website – sequence: 2 dbid: 8FG name: ProQuest Technology Collection url: https://search.proquest.com/technologycollection1 sourceTypes: Aggregation Database |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Engineering |
EISSN | 2057-3960 |
EndPage | 9 |
ExternalDocumentID | oai_doaj_org_article_c8ff8838611448319aa4e6305809ec41 10_1038_s41524_022_00794_8 |
GroupedDBID | 0R~ 3V. 5VS 7X7 8FE 8FG 8FH 8FI 8FJ AAJSJ AAYXX ABJCF ABUWG ACGFS ACSMW ADBBV AFKRA AJTQC ALIPV ALMA_UNASSIGNED_HOLDINGS ARCSS BBNVY BCNDV BENPR BGLVJ BHPHI BPHCQ BVXVI C6C CCPQU CITATION D1I EBLON EBS FYUFA GROUPED_DOAJ HCIFZ HMCUK KB. KQ8 LK8 M7P M~E NAO NO~ OK1 PDBOC PIMPY PQQKQ PROAC RNT SNYQT UKHRP 7XB 8FK AZQEC DWQXO GNUQQ K9. PQEST PQUKI PRINS |
ID | FETCH-LOGICAL-c385t-75d8a6b16ab096a73b5e3d4399096b92f390a7dc765d6cd05f7d9458fbc9b3533 |
IEDL.DBID | 7X7 |
ISSN | 2057-3960 |
IngestDate | Sun Sep 29 07:27:24 EDT 2024 Fri Sep 13 00:24:04 EDT 2024 Wed Oct 02 14:40:04 EDT 2024 |
IsDoiOpenAccess | true |
IsOpenAccess | true |
IsPeerReviewed | true |
IsScholarly | true |
Issue | 1 |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-c385t-75d8a6b16ab096a73b5e3d4399096b92f390a7dc765d6cd05f7d9458fbc9b3533 |
ORCID | 0000-0001-6096-2031 0000-0002-4911-0046 0000-0003-2229-6730 |
OpenAccessLink | https://www.proquest.com/docview/2667090912/abstract/?pq-origsite=%requestingapplication% |
PQID | 2667090912 |
PQPubID | 2041924 |
PageCount | 9 |
ParticipantIDs | doaj_primary_oai_doaj_org_article_c8ff8838611448319aa4e6305809ec41 proquest_journals_2667090912 crossref_primary_10_1038_s41524_022_00794_8 |
PublicationCentury | 2000 |
PublicationDate | 2022-05-20 |
PublicationDateYYYYMMDD | 2022-05-20 |
PublicationDate_xml | – month: 05 year: 2022 text: 2022-05-20 day: 20 |
PublicationDecade | 2020 |
PublicationPlace | London |
PublicationPlace_xml | – name: London |
PublicationTitle | npj computational materials |
PublicationYear | 2022 |
Publisher | Nature Publishing Group Nature Portfolio |
Publisher_xml | – name: Nature Publishing Group – name: Nature Portfolio |
References | 794_CR20 L Breiman (794_CR9) 2001; 45 M De Jong (794_CR37) 2015; 2 R Jacobs (794_CR41) 2020; 176 L Hirschfeld (794_CR1) 2020; 60 D Morgan (794_CR19) 2020; 50 D Schwalbe-Koda (794_CR6) 2021; 12 CFJ Wu (794_CR15) 1986; 14 HJ Lu (794_CR26) 2019; 169 W Li (794_CR28) 2018; 150 C Wen (794_CR31) 2019; 170 R Yuan (794_CR30) 2018; 30 794_CR17 G Pilania (794_CR38) 2016; 6 B Efron (794_CR11) 1983; 37 D Levi (794_CR22) 2019; 1905 JC Platt (794_CR21) 1999; 10 J Busk (794_CR23) 2022; 3 B Lu (794_CR18) 2021; 22 J Sexton (794_CR16) 2009; 53 794_CR13 794_CR35 M de Jong (794_CR36) 2015; 2 JP Janet (794_CR2) 2019; 10 B Efron (794_CR12) 1997; 92 794_CR34 F Musil (794_CR24) 2019; 15 V Stanev (794_CR39) 2018; 4 S Wager (794_CR10) 2014; 15 Y Tian (794_CR5) 2020; 128 R Liu (794_CR3) 2019; 59 P Virtanen (794_CR40) 2020; 17 DR Jones (794_CR32) 1998; 13 794_CR7 K Tran (794_CR4) 2020; 1 AM Molinaro (794_CR14) 2005; 21 J Ling (794_CR25) 2017; 6 794_CR29 F Pedregosa (794_CR33) 2011; 12 794_CR8 JH Friedman (794_CR27) 1991; 19 |
References_xml | – volume: 45 start-page: 5 year: 2001 ident: 794_CR9 publication-title: Mach. Learn doi: 10.1023/A:1010933404324 contributor: fullname: L Breiman – volume: 10 start-page: 61 year: 1999 ident: 794_CR21 publication-title: Methods Adv. Large Margin Classif. contributor: fullname: JC Platt – volume: 12 start-page: 1 year: 2021 ident: 794_CR6 publication-title: Nat. Commun. doi: 10.1038/s41467-021-25342-8 contributor: fullname: D Schwalbe-Koda – volume: 17 start-page: 261 year: 2020 ident: 794_CR40 publication-title: Nat. Methods doi: 10.1038/s41592-019-0686-2 contributor: fullname: P Virtanen – volume: 22 start-page: 1 year: 2021 ident: 794_CR18 publication-title: J. Mach. Learn. Res. contributor: fullname: B Lu – volume: 4 start-page: 1 year: 2018 ident: 794_CR39 publication-title: npj Comput. Mater doi: 10.1038/s41524-018-0085-8 contributor: fullname: V Stanev – volume: 92 start-page: 548 year: 1997 ident: 794_CR12 publication-title: J. Am. Stat. Assoc. contributor: fullname: B Efron – volume: 10 start-page: 7913 year: 2019 ident: 794_CR2 publication-title: Chem. Sci. doi: 10.1039/C9SC02298H contributor: fullname: JP Janet – volume: 50 start-page: 71 year: 2020 ident: 794_CR19 publication-title: Annu. Rev. Mater. Res. doi: 10.1146/annurev-matsci-070218-010015 contributor: fullname: D Morgan – ident: 794_CR35 doi: 10.1038/nn.3331 – volume: 30 start-page: 1 year: 2018 ident: 794_CR30 publication-title: Adv. Mater. doi: 10.1002/adma.201702884 contributor: fullname: R Yuan – volume: 169 start-page: 109075 year: 2019 ident: 794_CR26 publication-title: Comput. Mater. Sci. doi: 10.1016/j.commatsci.2019.06.010 contributor: fullname: HJ Lu – volume: 37 start-page: 36 year: 1983 ident: 794_CR11 publication-title: Am. Stat. doi: 10.1080/00031305.1983.10483087 contributor: fullname: B Efron – volume: 60 start-page: 3770 year: 2020 ident: 794_CR1 publication-title: J. Chem. Inf. Model. doi: 10.1021/acs.jcim.0c00502 contributor: fullname: L Hirschfeld – volume: 15 start-page: 1625 year: 2014 ident: 794_CR10 publication-title: J. Mach. Learn. Res. contributor: fullname: S Wager – ident: 794_CR7 doi: 10.1002/9781119148739.ch4 – volume: 19 start-page: 1 year: 1991 ident: 794_CR27 publication-title: Ann. Stat. contributor: fullname: JH Friedman – volume: 53 start-page: 801 year: 2009 ident: 794_CR16 publication-title: Comput. Stat. Data Anal. doi: 10.1016/j.csda.2008.08.007 contributor: fullname: J Sexton – ident: 794_CR34 – volume: 21 start-page: 3301 year: 2005 ident: 794_CR14 publication-title: Bioinformatics doi: 10.1093/bioinformatics/bti499 contributor: fullname: AM Molinaro – volume: 2 year: 2015 ident: 794_CR37 publication-title: Sci. Data doi: 10.1038/sdata.2015.9 contributor: fullname: M De Jong – volume: 14 start-page: 1261 year: 1986 ident: 794_CR15 publication-title: Ann. Stat. contributor: fullname: CFJ Wu – ident: 794_CR17 doi: 10.1111/j.2517-6161.1992.tb01866.x – volume: 3 start-page: 015012 year: 2022 ident: 794_CR23 publication-title: Mach. Learn. Sci. Technol. doi: 10.1088/2632-2153/ac3eb3 contributor: fullname: J Busk – volume: 1905 start-page: 11659 year: 2019 ident: 794_CR22 publication-title: ArXiv Prepr. contributor: fullname: D Levi – volume: 176 start-page: 109544 year: 2020 ident: 794_CR41 publication-title: Comput. Mater. Sci. doi: 10.1016/j.commatsci.2020.109544 contributor: fullname: R Jacobs – volume: 15 start-page: 906 year: 2019 ident: 794_CR24 publication-title: J. Chem. Theory Comput. doi: 10.1021/acs.jctc.8b00959 contributor: fullname: F Musil – volume: 170 start-page: 109 year: 2019 ident: 794_CR31 publication-title: Acta Mater. doi: 10.1016/j.actamat.2019.03.010 contributor: fullname: C Wen – ident: 794_CR20 – volume: 6 start-page: 207 year: 2017 ident: 794_CR25 publication-title: Integr. Mater. Manuf. Innov. doi: 10.1007/s40192-017-0098-z contributor: fullname: J Ling – volume: 150 start-page: 454 year: 2018 ident: 794_CR28 publication-title: Comput. Mater. Sci. doi: 10.1016/j.commatsci.2018.04.033 contributor: fullname: W Li – volume: 12 start-page: 2825 year: 2011 ident: 794_CR33 publication-title: J. Mach. Learn. Res contributor: fullname: F Pedregosa – volume: 1 start-page: 025006 year: 2020 ident: 794_CR4 publication-title: Mach. Learn. Sci. Technol. doi: 10.1088/2632-2153/ab7e1a contributor: fullname: K Tran – ident: 794_CR8 doi: 10.1007/3-540-45014-9_1 – volume: 6 year: 2016 ident: 794_CR38 publication-title: Sci. Rep. doi: 10.1038/srep19375 contributor: fullname: G Pilania – volume: 59 start-page: 181 year: 2019 ident: 794_CR3 publication-title: J. Chem. Inf. Model. doi: 10.1021/acs.jcim.8b00597 contributor: fullname: R Liu – ident: 794_CR13 – volume: 2 year: 2015 ident: 794_CR36 publication-title: Sci. Data doi: 10.1038/sdata.2015.53 contributor: fullname: M de Jong – volume: 128 start-page: 014103 year: 2020 ident: 794_CR5 publication-title: J. Appl. Phys. doi: 10.1063/5.0012405 contributor: fullname: Y Tian – volume: 13 start-page: 455 year: 1998 ident: 794_CR32 publication-title: J. Glob. Optim. doi: 10.1023/A:1008306431147 contributor: fullname: DR Jones – ident: 794_CR29 doi: 10.1007/978-1-4614-7138-7 |
SSID | ssj0001928240 |
Score | 2.3578098 |
Snippet | Abstract Obtaining accurate estimates of machine learning model uncertainties on newly predicted data is essential for understanding the accuracy of the model... Obtaining accurate estimates of machine learning model uncertainties on newly predicted data is essential for understanding the accuracy of the model and... |
SourceID | doaj proquest crossref |
SourceType | Open Website Aggregation Database |
StartPage | 1 |
SubjectTerms | Accuracy Calibration Estimates Learning algorithms Machine learning Materials science Methods Model accuracy Neural networks Regression analysis Regression models Standard deviation Statistical analysis Statistical methods Uncertainty |
SummonAdditionalLinks | – databaseName: DOAJ Directory of Open Access Journals dbid: DOA link: http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV07T8MwELZQJxgQT1EoyAMbspqHXxkBUVVIMFGpm-UngiGUNh3495ztgIoYWBgTJUry3dn3XXz-DqHLorK1YCKQKDVCqHMNkc5wGFescpW0gad9aw-PfDqj93M232j1FWvCsjxwBm5sZQhS1pIDcacSHEZr6jl4qSwab2lOfEq2kUy9Zt4iIVb1u2TgNcarGKkoicXrEBYbSuSPSJQE-3_NxynITPbQbs8O8XV-q3205dsDtLOhGXiI5nE3lcl2w6nDNwai3MU_FgsMDBRra9dR_wFDxMrr_d0Hfl_rXBWU73tp8dI_5xLYFqduOKsjNJvcPd1OSd8egdhaso4I5qTmpuTaQB6iRW2Yr13ML-DQNFWom0ILZwVnjltXsCBcQ5kMxjamBpp3jAbtW-tPEGYBklNXGAaw0ZIKrblj1DYeshMvrR2iqy-o1CKrYKi0el1LlYFVAKxKwCo5RDcRze8ro4J1OgF2Vb1d1V92HaLRly1UP6xWCtiEKODzyur0P55xhrar5BAMJosRGnTLtT8HjtGZi-ROn7Xky-o priority: 102 providerName: Directory of Open Access Journals |
Title | Calibration after bootstrap for accurate uncertainty quantification in regression models |
URI | https://www.proquest.com/docview/2667090912/abstract/ https://doaj.org/article/c8ff8838611448319aa4e6305809ec41 |
Volume | 8 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1La9tAEB7yuDSHkL6om9TsIbewWJb2eQpOiGsMCSVtwLdlXwq9yI4tH_rvM7uSk5ZALxJ6gZjZnW--3XkAnBelrySXNU2lRigLQVMVnMB5xctQKl-LnLd2eydmD2y-4Is9mO1yYVJY5c4mZkMdlj6tkY8QSGShEd3KkXVpFcC3o8vVE039o9I-a99MYx8OxyW6FTiy5UK-rrZopBas6LNm8LdGm4RcjKZgdoRJzaj6B5lyAf839jmDzvQEjntvkUw69b6Hvdh8gKO_agh-hEXKrnKdHknu-E3QcW7Tv68IeqTEer9N9SAIIli3_9_-IU9b20UJdd_9bsg6PnYhsQ3J3XE2n-BhevPrekb7dgnUV4q3VPKgrHBjYR3yEisrx2MVEt_AS6fLutKFlcFLwYPwoeC1DJpxVTuvXYVu32c4aJZN_AKE10hWQ-E4io2NmbRWBM68jshWovJ-ABc7UZlVVxXD5N3sSplOsAYFa7JgjRrAVZLmy5uponW-sVw_mn6CGK_qWqlKCSRoTKFhsJZFgdZIFTp6Nh7A2U4Xpp9mG_M6KL7-__EpvCuzqjmahTM4aNfb-A29idYN80DBo5p-H8LhZDL_Ocfz1c3dj_thZujP2qrNcw |
link.rule.ids | 315,786,790,870,2115,12083,12792,21416,27957,27958,31754,33408,33779,43345,43635,43840,74102,74392,74659 |
linkProvider | ProQuest |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1Lb9NAEF7RcKAcUHmpKS3dAze0iuN9nypARKG0OSWSb6t9OeLipLFz4N8zu3YIqFKPfknWzM588x6EPhWlp5LLmqRRI4SFoIkKToBc8TKUytci963dL8R8xW4rXg0Bt3YoqzzoxKyow8anGPkEgEQWGtCtvNk-kLQ1KmVXhxUaJ-g5o5Slkj5ZyWOMRYNDwYqhVwZ-ZtImvGIklbADOGpG1H94lMf2P9LKGWpmZ-jVYCPiLz1TX6NnsXmDXv4zOfAtqlJPleu5h_OebwzmcpfiFlsMdii23u_TFAgMuNVn_bvf-GFv-9qg_rtfDd7FdV8I2-C8E6d9h1az78tvczIsSSCeKt4RyYOywk2FdeCNWEkdjzQkLwMunS5rqgsrg5eCB-FDwWsZNOOqdl47CsbeezRqNk08R5jX4KKGwnEgG5syaa0InHkdwUeJyvsx-nwgldn2szBMzmFTZXrCGiCsyYQ1aoy-Jmr-fTPNsc43Nru1GcTCeFXXSlElwC1jCtSBtSwK0EGq0NGz6RhdHnhhBuFqzfEoXDz9-Bq9mC_v78zdj8XPD-i0zGznoBgu0ajb7eMV2BOd-5gPzR9hoMel |
linkToPdf | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1LTxsxELZaIqH2gFpo1QBtfeCGrGx2_Tyh0hKlrwghkHKz_ERcNiHZHPj3jL1OoarU476k1cx4vhn7mxmETqraNYKJSFKrEUK9V0R6y2FdsdrX0kWe69Z-z_j0hv6Ys3nhP60LrXLrE7Oj9guX9shHACSiUoBu9SgWWsTlt8nZ8p6kCVLppLWM03iJBoJyBhY-OL-YXV497bgoSC9oVSpn4NdG64RelCRCO0ClokT-hU65if8_PjoDz-QN2isRI_7Sq_gtehHaffT6WR_BAzRPFVa21yXOU78xBM9d2sVYYohKsXFuk3pCYECxngPQPeD7jemZQv13dy1ehdueFtviPCFn_Q7dTC6uv05JGZlAXCNZRwTz0nA75sZCbmJEY1lofMo54NKqOjaqMsI7wZnnzlcsCq8ok9E6ZRsI_d6jnXbRhg8IswgJq68sA7HRMRXGcM-oUwEyliCdG6LTraj0su-MofOJdiN1L1gNgtVZsFoO0XmS5p83U1frfGOxutVlkWgnY5SykRySNCrBORhDAwePJCsVHB0P0fFWF7ostbV-MozD_z_-jHbBYvSv77OfR-hVnbXOwEsco51utQkfIbjo7KdiNY8m381I |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Calibration+after+bootstrap+for+accurate+uncertainty+quantification+in+regression+models&rft.jtitle=npj+computational+materials&rft.au=Palmer%2C+Glenn&rft.au=Du+Siqi&rft.au=Politowicz+Alexander&rft.au=Emory%2C+Joshua+Paul&rft.date=2022-05-20&rft.pub=Nature+Publishing+Group&rft.eissn=2057-3960&rft.volume=8&rft.issue=1&rft_id=info:doi/10.1038%2Fs41524-022-00794-8&rft.externalDBID=HAS_PDF_LINK |
thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2057-3960&client=summon |
thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2057-3960&client=summon |
thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2057-3960&client=summon |