Flow Annealed Importance Sampling Bootstrap

Normalizing flows are tractable density models that can approximate complicated target distributions, e.g. Boltzmann distributions of physical systems. However, current methods for training flows either suffer from mode-seeking behavior, use samples from the target generated beforehand by expensive...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Laurence Illing Midgley, Stimper, Vincent, Simm, Gregor N C, Schölkopf, Bernhard, José Miguel Hernández-Lobato
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 07.03.2023
Subjects
Online AccessGet full text

Cover

Loading…
Abstract Normalizing flows are tractable density models that can approximate complicated target distributions, e.g. Boltzmann distributions of physical systems. However, current methods for training flows either suffer from mode-seeking behavior, use samples from the target generated beforehand by expensive MCMC methods, or use stochastic losses that have high variance. To avoid these problems, we augment flows with annealed importance sampling (AIS) and minimize the mass-covering \(\alpha\)-divergence with \(\alpha=2\), which minimizes importance weight variance. Our method, Flow AIS Bootstrap (FAB), uses AIS to generate samples in regions where the flow is a poor approximation of the target, facilitating the discovery of new modes. We apply FAB to multimodal targets and show that we can approximate them very accurately where previous methods fail. To the best of our knowledge, we are the first to learn the Boltzmann distribution of the alanine dipeptide molecule using only the unnormalized target density, without access to samples generated via Molecular Dynamics (MD) simulations: FAB produces better results than training via maximum likelihood on MD samples while using 100 times fewer target evaluations. After reweighting the samples, we obtain unbiased histograms of dihedral angles that are almost identical to the ground truth.
AbstractList Normalizing flows are tractable density models that can approximate complicated target distributions, e.g. Boltzmann distributions of physical systems. However, current methods for training flows either suffer from mode-seeking behavior, use samples from the target generated beforehand by expensive MCMC methods, or use stochastic losses that have high variance. To avoid these problems, we augment flows with annealed importance sampling (AIS) and minimize the mass-covering \(\alpha\)-divergence with \(\alpha=2\), which minimizes importance weight variance. Our method, Flow AIS Bootstrap (FAB), uses AIS to generate samples in regions where the flow is a poor approximation of the target, facilitating the discovery of new modes. We apply FAB to multimodal targets and show that we can approximate them very accurately where previous methods fail. To the best of our knowledge, we are the first to learn the Boltzmann distribution of the alanine dipeptide molecule using only the unnormalized target density, without access to samples generated via Molecular Dynamics (MD) simulations: FAB produces better results than training via maximum likelihood on MD samples while using 100 times fewer target evaluations. After reweighting the samples, we obtain unbiased histograms of dihedral angles that are almost identical to the ground truth.
Author José Miguel Hernández-Lobato
Simm, Gregor N C
Laurence Illing Midgley
Stimper, Vincent
Schölkopf, Bernhard
Author_xml – sequence: 1
  fullname: Laurence Illing Midgley
– sequence: 2
  givenname: Vincent
  surname: Stimper
  fullname: Stimper, Vincent
– sequence: 3
  givenname: Gregor
  surname: Simm
  middlename: N C
  fullname: Simm, Gregor N C
– sequence: 4
  givenname: Bernhard
  surname: Schölkopf
  fullname: Schölkopf, Bernhard
– sequence: 5
  fullname: José Miguel Hernández-Lobato
BookMark eNrjYmDJy89LZWLgNDI2NtS1MDEy4mDgLS7OMjAwMDIzNzI1NeZk0HbLyS9XcMzLS03MSU1R8MwtyC8qScxLTlUITswtyMnMS1dwys8vKS4pSizgYWBNS8wpTuWF0twMym6uIc4eugVF-YWlqcUl8Vn5pUV5QKl4IzNLC0sLI3NjU2PiVAEAcGkzHw
ContentType Paper
Copyright 2023. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Copyright_xml – notice: 2023. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
DBID 8FE
8FG
ABJCF
ABUWG
AFKRA
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
HCIFZ
L6V
M7S
PIMPY
PQEST
PQQKQ
PQUKI
PRINS
PTHSS
DatabaseName ProQuest SciTech Collection
ProQuest Technology Collection
Materials Science & Engineering Collection
ProQuest Central (Alumni)
ProQuest Central
ProQuest Central Essentials
ProQuest Central
Technology Collection
ProQuest One Community College
ProQuest Central Korea
SciTech Premium Collection
ProQuest Engineering Collection
Engineering Database
Publicly Available Content (ProQuest)
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Academic
ProQuest One Academic UKI Edition
ProQuest Central China
Engineering Collection
DatabaseTitle Publicly Available Content Database
Engineering Database
Technology Collection
ProQuest Central Essentials
ProQuest One Academic Eastern Edition
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest Technology Collection
ProQuest SciTech Collection
ProQuest Central China
ProQuest Central
ProQuest Engineering Collection
ProQuest One Academic UKI Edition
ProQuest Central Korea
Materials Science & Engineering Collection
ProQuest One Academic
Engineering Collection
DatabaseTitleList Publicly Available Content Database
Database_xml – sequence: 1
  dbid: 8FG
  name: ProQuest Technology Collection
  url: https://search.proquest.com/technologycollection1
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Physics
EISSN 2331-8422
Genre Working Paper/Pre-Print
GroupedDBID 8FE
8FG
ABJCF
ABUWG
AFKRA
ALMA_UNASSIGNED_HOLDINGS
AZQEC
BENPR
BGLVJ
CCPQU
DWQXO
FRJ
HCIFZ
L6V
M7S
M~E
PIMPY
PQEST
PQQKQ
PQUKI
PRINS
PTHSS
ID FETCH-proquest_journals_26989827353
IEDL.DBID 8FG
IngestDate Thu Oct 10 16:29:14 EDT 2024
IsOpenAccess true
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-proquest_journals_26989827353
OpenAccessLink https://www.proquest.com/docview/2698982735?pq-origsite=%requestingapplication%
PQID 2698982735
PQPubID 2050157
ParticipantIDs proquest_journals_2698982735
PublicationCentury 2000
PublicationDate 20230307
PublicationDateYYYYMMDD 2023-03-07
PublicationDate_xml – month: 03
  year: 2023
  text: 20230307
  day: 07
PublicationDecade 2020
PublicationPlace Ithaca
PublicationPlace_xml – name: Ithaca
PublicationTitle arXiv.org
PublicationYear 2023
Publisher Cornell University Library, arXiv.org
Publisher_xml – name: Cornell University Library, arXiv.org
SSID ssj0002672553
Score 3.4545817
SecondaryResourceType preprint
Snippet Normalizing flows are tractable density models that can approximate complicated target distributions, e.g. Boltzmann distributions of physical systems....
SourceID proquest
SourceType Aggregation Database
SubjectTerms Alanine
Annealing
Boltzmann distribution
Density
Histograms
Importance sampling
Molecular dynamics
Normalizing
Training
Title Flow Annealed Importance Sampling Bootstrap
URI https://www.proquest.com/docview/2698982735
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwY2BQSbEwTTRJSTTSBTZFDIAdFBNT3cTUNBNdw1SzFFPDNCNLs2TQeIevn5lHqIlXhGkEdMCtGLqsElYmggvqlPxk0Bi5vhH4pkNgZWtqX1CoC7o1CjS7Cr1Cg5mB1dDI3Ay0pM_CzR0-xmJkZg5sMRtjFLPgusNNkIE1ILEgtUiIgSk1T5iBHbzkMrlYhEHbLSe_XMERWM4By-gUBc9ccEsYGAcKwYmgZd556QpO-fkloKGIAlEGZTfXEGcPXZgN8dA0UByPcLGxGAMLsDOfKsGgYGiRaJBsaJpiAQwkk8SkVMsUw5RkcwPDlCRg1yAx0UySQQafSVL4paUZuEDXoYPXSJnLMLCUFJWmygIrzZIkOXDIyDGwOrn6BQQBeb51rgBUxHa5
link.rule.ids 780,784,12765,21388,33373,33744,43600,43805
linkProvider ProQuest
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV3PS8MwFH7ohribP9E5NaA3CTZtknYnUbF2ug3BCbuVtMl20bWuFf99X0KnB2HnQBJewve99-WFD-BSR0JxrXyKqYiHBQoXVJkZp8xILdjM78vc6h2jsUze-NNUTBvBrWraKleY6IBaF7nVyK9953SIZCtuyk9qXaPs62pjobEJbR4gdduf4vHjr8biyxAz5uAfzDruiHeg_aJKs9yFDbPYgy3XcplX-3AVvxff5BZxDjFak8GHy4TxDMirsm3eizm5K4raShHlAVzED5P7hK5WSJs7UKV_Ow4OoYXFvDkCwiLl5UzoCIPEVWb6muk89JjOsDRQSh5Db91M3fXD57CdTEbDdDgYP59Ax1qju36psAetevllTpFA6-zMRekHc4Z20A
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Flow+Annealed+Importance+Sampling+Bootstrap&rft.jtitle=arXiv.org&rft.au=Laurence+Illing+Midgley&rft.au=Stimper%2C+Vincent&rft.au=Simm%2C+Gregor+N+C&rft.au=Sch%C3%B6lkopf%2C+Bernhard&rft.date=2023-03-07&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422