Wasserstein Dictionary Learning: Optimal Transport-based unsupervised non-linear dictionary learning
This paper introduces a new nonlinear dictionary learning method for histograms in the probability simplex. The method leverages optimal transport theory, in the sense that our aim is to reconstruct histograms using so-called displacement interpolations (a.k.a. Wasserstein barycenters) between dicti...
Saved in:
Published in | arXiv.org |
---|---|
Main Authors | , , , , , , , |
Format | Paper Journal Article |
Language | English |
Published |
Ithaca
Cornell University Library, arXiv.org
15.03.2018
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | This paper introduces a new nonlinear dictionary learning method for histograms in the probability simplex. The method leverages optimal transport theory, in the sense that our aim is to reconstruct histograms using so-called displacement interpolations (a.k.a. Wasserstein barycenters) between dictionary atoms; such atoms are themselves synthetic histograms in the probability simplex. Our method simultaneously estimates such atoms, and, for each datapoint, the vector of weights that can optimally reconstruct it as an optimal transport barycenter of such atoms. Our method is computationally tractable thanks to the addition of an entropic regularization to the usual optimal transportation problem, leading to an approximation scheme that is efficient, parallel and simple to differentiate. Both atoms and weights are learned using a gradient-based descent method. Gradients are obtained by automatic differentiation of the generalized Sinkhorn iterations that yield barycenters with entropic smoothing. Because of its formulation relying on Wasserstein barycenters instead of the usual matrix product between dictionary and codes, our method allows for nonlinear relationships between atoms and the reconstruction of input data. We illustrate its application in several different image processing settings. |
---|---|
AbstractList | SIAM Journal on Imaging Sciences 11(1) (2018) 643-678 This paper introduces a new nonlinear dictionary learning method for
histograms in the probability simplex. The method leverages optimal transport
theory, in the sense that our aim is to reconstruct histograms using so-called
displacement interpolations (a.k.a. Wasserstein barycenters) between dictionary
atoms; such atoms are themselves synthetic histograms in the probability
simplex. Our method simultaneously estimates such atoms, and, for each
datapoint, the vector of weights that can optimally reconstruct it as an
optimal transport barycenter of such atoms. Our method is computationally
tractable thanks to the addition of an entropic regularization to the usual
optimal transportation problem, leading to an approximation scheme that is
efficient, parallel and simple to differentiate. Both atoms and weights are
learned using a gradient-based descent method. Gradients are obtained by
automatic differentiation of the generalized Sinkhorn iterations that yield
barycenters with entropic smoothing. Because of its formulation relying on
Wasserstein barycenters instead of the usual matrix product between dictionary
and codes, our method allows for nonlinear relationships between atoms and the
reconstruction of input data. We illustrate its application in several
different image processing settings. This paper introduces a new nonlinear dictionary learning method for histograms in the probability simplex. The method leverages optimal transport theory, in the sense that our aim is to reconstruct histograms using so-called displacement interpolations (a.k.a. Wasserstein barycenters) between dictionary atoms; such atoms are themselves synthetic histograms in the probability simplex. Our method simultaneously estimates such atoms, and, for each datapoint, the vector of weights that can optimally reconstruct it as an optimal transport barycenter of such atoms. Our method is computationally tractable thanks to the addition of an entropic regularization to the usual optimal transportation problem, leading to an approximation scheme that is efficient, parallel and simple to differentiate. Both atoms and weights are learned using a gradient-based descent method. Gradients are obtained by automatic differentiation of the generalized Sinkhorn iterations that yield barycenters with entropic smoothing. Because of its formulation relying on Wasserstein barycenters instead of the usual matrix product between dictionary and codes, our method allows for nonlinear relationships between atoms and the reconstruction of input data. We illustrate its application in several different image processing settings. |
Author | Heitz, Matthieu Bonneel, Nicolas Cuturi, Marco Starck, Jean-Luc Schmitz, Morgan A Coeurjolly, David Fred Maurice Ngolè Mboula Peyré, Gabriel |
Author_xml | – sequence: 1 givenname: Morgan surname: Schmitz middlename: A fullname: Schmitz, Morgan A – sequence: 2 givenname: Matthieu surname: Heitz fullname: Heitz, Matthieu – sequence: 3 givenname: Nicolas surname: Bonneel fullname: Bonneel, Nicolas – sequence: 4 fullname: Fred Maurice Ngolè Mboula – sequence: 5 givenname: David surname: Coeurjolly fullname: Coeurjolly, David – sequence: 6 givenname: Marco surname: Cuturi fullname: Cuturi, Marco – sequence: 7 givenname: Gabriel surname: Peyré fullname: Peyré, Gabriel – sequence: 8 givenname: Jean-Luc surname: Starck fullname: Starck, Jean-Luc |
BackLink | https://doi.org/10.1137/17M1140431$$DView published paper (Access to full text may be restricted) https://doi.org/10.48550/arXiv.1708.01955$$DView paper in arXiv |
BookMark | eNpFkE1Lw0AQhhdRsNb-AE8GPKdO9iO78Sa1fkCgl4DHMJtsZEvcxN2k2H9v-gGehhneeXh4b8il65wh5C6BJVdCwCP6X7tbJhLUEpJMiAsyo4wlseKUXpNFCFsAoKmkQrAZqT8xBOPDYKyLXmw12M6h30e5Qe-s-3qKNv1gv7GNCo8u9J0fYo3B1NHowtgbv7OHZZKIW-ump6j-h7RnyC25arANZnGec1K8rovVe5xv3j5Wz3mMgmZxk4JqUAmTMS6V1oxiJUGL6cITw4WCDFml0xoUctRYNSAlawB5yitNGzYn9yfssYKy95O335eHKspjFVPi4ZToffczmjCU2270bnIqKchEpkqlGfsDnwFliQ |
ContentType | Paper Journal Article |
Copyright | 2018. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. http://arxiv.org/licenses/nonexclusive-distrib/1.0 |
Copyright_xml | – notice: 2018. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. – notice: http://arxiv.org/licenses/nonexclusive-distrib/1.0 |
DBID | 8FE 8FG ABJCF ABUWG AFKRA AZQEC BENPR BGLVJ CCPQU DWQXO HCIFZ L6V M7S PIMPY PQEST PQQKQ PQUKI PRINS PTHSS AKY AKZ EPD GOX |
DOI | 10.48550/arxiv.1708.01955 |
DatabaseName | ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central (Alumni) ProQuest Central UK/Ireland ProQuest Central Essentials AUTh Library subscriptions: ProQuest Central Technology Collection ProQuest One Community College ProQuest Central SciTech Premium Collection (Proquest) (PQ_SDU_P3) ProQuest Engineering Collection ProQuest Engineering Database Access via ProQuest (Open Access) ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection arXiv Computer Science arXiv Mathematics arXiv Statistics arXiv.org |
DatabaseTitle | Publicly Available Content Database Engineering Database Technology Collection ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Technology Collection ProQuest SciTech Collection ProQuest Central China ProQuest Central ProQuest Engineering Collection ProQuest One Academic UKI Edition ProQuest Central Korea Materials Science & Engineering Collection ProQuest One Academic Engineering Collection |
DatabaseTitleList | Publicly Available Content Database |
Database_xml | – sequence: 1 dbid: GOX name: arXiv.org url: http://arxiv.org/find sourceTypes: Open Access Repository – sequence: 2 dbid: 8FG name: ProQuest Technology Collection url: https://search.proquest.com/technologycollection1 sourceTypes: Aggregation Database |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Physics |
EISSN | 2331-8422 |
ExternalDocumentID | 1708_01955 |
Genre | Working Paper/Pre-Print |
GroupedDBID | 8FE 8FG ABJCF ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS AZQEC BENPR BGLVJ CCPQU DWQXO FRJ HCIFZ L6V M7S M~E PIMPY PQEST PQQKQ PQUKI PRINS PTHSS AKY AKZ EPD GOX |
ID | FETCH-LOGICAL-a529-f608fa85e93478bb32ac70b585e41e45809a3cb6d08a4abacf0773f0a464cb2f3 |
IEDL.DBID | 8FG |
IngestDate | Mon Jan 08 05:44:11 EST 2024 Thu Oct 10 20:39:32 EDT 2024 |
IsDoiOpenAccess | true |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-a529-f608fa85e93478bb32ac70b585e41e45809a3cb6d08a4abacf0773f0a464cb2f3 |
OpenAccessLink | https://www.proquest.com/docview/2071768869?pq-origsite=%requestingapplication% |
PQID | 2071768869 |
PQPubID | 2050157 |
ParticipantIDs | arxiv_primary_1708_01955 proquest_journals_2071768869 |
PublicationCentury | 2000 |
PublicationDate | 20180315 |
PublicationDateYYYYMMDD | 2018-03-15 |
PublicationDate_xml | – month: 03 year: 2018 text: 20180315 day: 15 |
PublicationDecade | 2010 |
PublicationPlace | Ithaca |
PublicationPlace_xml | – name: Ithaca |
PublicationTitle | arXiv.org |
PublicationYear | 2018 |
Publisher | Cornell University Library, arXiv.org |
Publisher_xml | – name: Cornell University Library, arXiv.org |
SSID | ssj0002672553 |
Score | 1.6967499 |
SecondaryResourceType | preprint |
Snippet | This paper introduces a new nonlinear dictionary learning method for histograms in the probability simplex. The method leverages optimal transport theory, in... SIAM Journal on Imaging Sciences 11(1) (2018) 643-678 This paper introduces a new nonlinear dictionary learning method for histograms in the probability... |
SourceID | arxiv proquest |
SourceType | Open Access Repository Aggregation Database |
SubjectTerms | Atomic properties Center of gravity Computer Science - Graphics Histograms Image processing Learning Mathematics - Optimization and Control Optimization Regularization Statistics - Machine Learning Transport theory Transportation problem |
SummonAdditionalLinks | – databaseName: arXiv.org dbid: GOX link: http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwdV09T8MwED21nVgQCFALBXlgtXASf7IhoFRI0KWIbpHt2FUHqiptEfx7bCelA2J0ZDvSXS73zr57B3BNvEicI9irzGGqqAsmVfGYRcisYZWn6TDn5ZWP3-jzjM06gHa1MLr-Wnw2_MBmfZOJmOqYKca60M3zmLL1NJk1l5OJiqudv58XMGZ69OfXmvzF6AgOW6CH7hrNHEPHLU-getepvDG2mEQPi1RToOtv1LKczm_RJJjwR1j3SzqOo5-p0Ha53q6iXcdBCNlxhIe6RtV-k7YBxPwUpqPH6f0Yt30OsGa5wp4T6bVkThVUSGOKXFtBTMDxjmaOMkmULqzhFZGaaqOtJ0IUnmjKqTW5L86gF97r-oAin5vwhSLeaqpCpCOEDRiEs8L4sKEZQD9Jp1w1VBZlFFyZBDeA4U5gZfsZr8s8RntcSq7O_195AQcBRaRCvYwNobept-4yeOqNuUrq-gHVwpUS priority: 102 providerName: Cornell University |
Title | Wasserstein Dictionary Learning: Optimal Transport-based unsupervised non-linear dictionary learning |
URI | https://www.proquest.com/docview/2071768869 https://arxiv.org/abs/1708.01955 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV3NS8MwFA-6Injzk03n6MFrNG3SpvEiqPtA2AcycbeSpMkY6KztJnrxbzfJuu0geCmkgQRekvd-eXnv9wC4RJo6zhGoWaAgYUSZI5XFNoowkiLKNHHOnP4g7j2Tx0k0qRxuZRVWudaJTlFn79L6yK0nJDDQOInZbf4BbdUo-7paldDYBV5gmfBspninu_GxhDE1iBmvHjMdddc1L75mn1cBtSGUAbMJfp779UcVO_vSOQDeiOeqOAQ7an4E9lxYpiyPQfbCXTqkLUnpP8xcDgIvvv2KFXV64w_NkX_jr_6GpBxau5T5y3m5zK0esA1zxYcWTvLCz7aDVAUjpidg3GmP73uwqosAeRQyqGOUaJ5EimFCEyFwyCVFwuB-RQJFogQxjqWIM5RwwgWXGlGKNeIkJlKEGp-CmplX1YFv-d-oxgxpyQkzNyNKpcEscYSFNgOKBqg76aT5ivoitYJLneAaoLkWWFpt-zLdLtLZ_93nYN8gD5fcF0RNUFsUS3VhrPtCtNwStoB31x6MnkyrO5yYb_-n_QsiQqma |
link.rule.ids | 228,230,786,790,891,12792,21416,27956,33406,33777,43633,43838 |
linkProvider | ProQuest |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV3PS8MwFA66InrzJ06n9uC1mjZp0ngR1I2p2xwycbeSpMkY6KztJvrfm2TddhA8toUUXpL3vry8930AnENNHedIoFmoAsywMlsqI7aKMJYizjR2yZxuj7Rf8MMwHlYJt7Iqq1z4ROeosw9pc-Q2ExIaaJwQdp1_BlY1yt6uVhIa68CzlJtJDXg3zV7_eZlliQg1mBnNrzMdedclL77HXxchtUWUIbMtfp579ccZuwjT2gZen-eq2AFrarILNlxhpiz3QPbKXUOkFaX078auC4EXP37Fizq68p_Mpn_nb_6SpjywkSnzZ5NylltPYB_MIT-wgJIXfrYapJKMGO2DQas5uG0HlTJCwOOIBZrARPMkVgxhmgiBIi4pFAb5KxwqHCeQcSQFyWDCMRdcakgp0pBjgqWINDoANfNfdQh8ywBHNWJQS46ZORtRKg1qITES2gwo6uDQWSfN5-QXqTVc6gxXB42FwdJq4ZfpapqO_v98Bjbbg24n7dz3Ho_BlsEhrtUvjBugNi1m6sTE-qk4rSb0F95Rqhc |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Wasserstein+Dictionary+Learning%3A+Optimal+Transport-based+unsupervised+non-linear+dictionary+learning&rft.jtitle=arXiv.org&rft.au=Schmitz%2C+Morgan+A&rft.au=Heitz%2C+Matthieu&rft.au=Bonneel%2C+Nicolas&rft.au=Fred+Maurice+Ngol%C3%A8+Mboula&rft.date=2018-03-15&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422&rft_id=info:doi/10.48550%2Farxiv.1708.01955 |