Heating up decision boundaries: isocapacitory saturation, adversarial scenarios and generalization bounds
In the present work we study classifiers' decision boundaries via Brownian motion processes in ambient data space and associated probabilistic techniques. Intuitively, our ideas correspond to placing a heat source at the decision boundary and observing how effectively the sample points warm up....
Saved in:
Published in | arXiv.org |
---|---|
Main Authors | , , |
Format | Paper |
Language | English |
Published |
Ithaca
Cornell University Library, arXiv.org
15.01.2021
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | In the present work we study classifiers' decision boundaries via Brownian motion processes in ambient data space and associated probabilistic techniques. Intuitively, our ideas correspond to placing a heat source at the decision boundary and observing how effectively the sample points warm up. We are largely motivated by the search for a soft measure that sheds further light on the decision boundary's geometry. En route, we bridge aspects of potential theory and geometric analysis (Mazya, 2011, Grigoryan-Saloff-Coste, 2002) with active fields of ML research such as adversarial examples and generalization bounds. First, we focus on the geometric behavior of decision boundaries in the light of adversarial attack/defense mechanisms. Experimentally, we observe a certain capacitory trend over different adversarial defense strategies: decision boundaries locally become flatter as measured by isoperimetric inequalities (Ford et al, 2019); however, our more sensitive heat-diffusion metrics extend this analysis and further reveal that some non-trivial geometry invisible to plain distance-based methods is still preserved. Intuitively, we provide evidence that the decision boundaries nevertheless retain many persistent "wiggly and fuzzy" regions on a finer scale. Second, we show how Brownian hitting probabilities translate to soft generalization bounds which are in turn connected to compression and noise stability (Arora et al, 2018), and these bounds are significantly stronger if the decision boundary has controlled geometric features. |
---|---|
AbstractList | In the present work we study classifiers' decision boundaries via Brownian motion processes in ambient data space and associated probabilistic techniques. Intuitively, our ideas correspond to placing a heat source at the decision boundary and observing how effectively the sample points warm up. We are largely motivated by the search for a soft measure that sheds further light on the decision boundary's geometry. En route, we bridge aspects of potential theory and geometric analysis (Mazya, 2011, Grigoryan-Saloff-Coste, 2002) with active fields of ML research such as adversarial examples and generalization bounds. First, we focus on the geometric behavior of decision boundaries in the light of adversarial attack/defense mechanisms. Experimentally, we observe a certain capacitory trend over different adversarial defense strategies: decision boundaries locally become flatter as measured by isoperimetric inequalities (Ford et al, 2019); however, our more sensitive heat-diffusion metrics extend this analysis and further reveal that some non-trivial geometry invisible to plain distance-based methods is still preserved. Intuitively, we provide evidence that the decision boundaries nevertheless retain many persistent "wiggly and fuzzy" regions on a finer scale. Second, we show how Brownian hitting probabilities translate to soft generalization bounds which are in turn connected to compression and noise stability (Arora et al, 2018), and these bounds are significantly stronger if the decision boundary has controlled geometric features. |
Author | Mukherjee, Mayukh Georgiev, Bogdan Franken, Lukas |
Author_xml | – sequence: 1 givenname: Bogdan surname: Georgiev fullname: Georgiev, Bogdan – sequence: 2 givenname: Lukas surname: Franken fullname: Franken, Lukas – sequence: 3 givenname: Mayukh surname: Mukherjee fullname: Mukherjee, Mayukh |
BookMark | eNqNjEsKwkAQRAdR8HuHBrcKcWI-uhXFA7iXNmllQuiO0xlBT29QD-CqHtSrGps-C1PPjGwcr5b52tqhmalWURTZNLNJEo-MOxK2jm8QGiipcOqE4SKBS_SOdAtOpcAGC9eKf4JiG3w3EF4Alg_y2mlYgxbEHYkCcgk3YvJYu9fH_N7p1AyuWCvNfjkx88P-tDsuGy_3QNqeKwmeu-ps11meZskmyuP_rDcDLEuU |
ContentType | Paper |
Copyright | 2021. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
Copyright_xml | – notice: 2021. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
DBID | 8FE 8FG ABJCF ABUWG AFKRA AZQEC BENPR BGLVJ CCPQU DWQXO HCIFZ L6V M7S PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
DatabaseName | ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central (Alumni Edition) ProQuest Central ProQuest Central Essentials AUTh Library subscriptions: ProQuest Central Technology Collection ProQuest One Community College ProQuest Central Korea SciTech Premium Collection ProQuest Engineering Collection Engineering Database Publicly Available Content Database ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection |
DatabaseTitle | Publicly Available Content Database Engineering Database Technology Collection ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Technology Collection ProQuest SciTech Collection ProQuest Central China ProQuest Central ProQuest Engineering Collection ProQuest One Academic UKI Edition ProQuest Central Korea Materials Science & Engineering Collection ProQuest One Academic Engineering Collection |
DatabaseTitleList | Publicly Available Content Database |
Database_xml | – sequence: 1 dbid: 8FG name: ProQuest Technology Collection url: https://search.proquest.com/technologycollection1 sourceTypes: Aggregation Database |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Physics |
EISSN | 2331-8422 |
Genre | Working Paper/Pre-Print |
GroupedDBID | 8FE 8FG ABJCF ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS AZQEC BENPR BGLVJ CCPQU DWQXO FRJ HCIFZ L6V M7S M~E PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
ID | FETCH-proquest_journals_24786759083 |
IEDL.DBID | 8FG |
IngestDate | Thu Oct 10 20:14:25 EDT 2024 |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-proquest_journals_24786759083 |
OpenAccessLink | https://www.proquest.com/docview/2478675908?pq-origsite=%requestingapplication% |
PQID | 2478675908 |
PQPubID | 2050157 |
ParticipantIDs | proquest_journals_2478675908 |
PublicationCentury | 2000 |
PublicationDate | 20210115 |
PublicationDateYYYYMMDD | 2021-01-15 |
PublicationDate_xml | – month: 01 year: 2021 text: 20210115 day: 15 |
PublicationDecade | 2020 |
PublicationPlace | Ithaca |
PublicationPlace_xml | – name: Ithaca |
PublicationTitle | arXiv.org |
PublicationYear | 2021 |
Publisher | Cornell University Library, arXiv.org |
Publisher_xml | – name: Cornell University Library, arXiv.org |
SSID | ssj0002672553 |
Score | 3.3155384 |
SecondaryResourceType | preprint |
Snippet | In the present work we study classifiers' decision boundaries via Brownian motion processes in ambient data space and associated probabilistic techniques.... |
SourceID | proquest |
SourceType | Aggregation Database |
SubjectTerms | Brownian motion Cost analysis Decision theory Potential theory |
Title | Heating up decision boundaries: isocapacitory saturation, adversarial scenarios and generalization bounds |
URI | https://www.proquest.com/docview/2478675908 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1LSwMxEB60RfDmEx-1BPRocLObbbJeBGXXRWgpotBbySZp6WVbm_bqbzeTbvUg9JYQGJJhmFdm5gO4s0qItKcnNOImpdwknEppJ9SISsVRIrJqgnnI_qBXfvK3UTpqEm6uKavc6sSgqM1cY478IeZCeuc2i-TT4osiahT-rjYQGvvQZjgJDzvFi9ffHEvcE95jTv6p2WA7iiNoD9XCLo9hz9YncBBKLrU7hVmJ7lo9JesFMQ3SDakCyhGGr49k5hnnTZme4T84cTiBM7DxnihEUXYKZYfgNCa_mjuiakOmmynSTXPlhpw7g9si_3gp6fZ-40aC3Pjvvck5tOp5bS-AWM64Zan2kaXlIpNKaCF1WlVMaaNidgmdXZSudh9fw2GMFRsRoyztQGu1XNsbb3JXVTfwtQvt53wwfPe7_nf-A10uj3Y |
link.rule.ids | 786,790,12792,21416,33406,33777,43633,43838 |
linkProvider | ProQuest |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1LSwMxEB60RfTmE6tVA3o0uI9kk3rxINZV2-KhQm9LNsmWvWxr0_5_M9utHoTeAoGQDMM8vpnMB3BnlRA80QUNmOGUmZhRKW1BjchVFMSilxeIQw5HSfrF3id80gBurmmr3NjE2lCbmUaM_CFiQvrgthfIp_k3RdYorK42FBq70GZxEqOey_7rL8YSJcJHzPE_M1v7jv4htD_V3C6OYMdWx7BXt1xqdwJliuFaNSWrOTEN0w3Ja5YjTF8fSekF512ZLrEOThxO4KzFeE8Usig7hbpDcBqTX80cUZUh0_UU6eZz5fo4dwq3_Zfxc0o398saDXLZ33vjM2hVs8qeA7EsZDbk2meWlomeVEILqXmeh0obFYUd6G476WL79g3sp-PhIBu8jT4u4SDC7o0gpCHvQmu5WNkr736X-XUt4x9RRI-T |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Heating+up+decision+boundaries%3A+isocapacitory+saturation%2C+adversarial+scenarios+and+generalization+bounds&rft.jtitle=arXiv.org&rft.au=Georgiev%2C+Bogdan&rft.au=Franken%2C+Lukas&rft.au=Mukherjee%2C+Mayukh&rft.date=2021-01-15&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422 |