Data-Free Sketch-Based Image Retrieval
Rising concerns about privacy and anonymity preservation of deep learning models have facilitated research in data-free learning (DFL). For the first time, we identify that for data-scarce tasks like Sketch-Based Image Retrieval (SBIR), where the difficulty in acquiring paired photos and hand-drawn...
Saved in:
Published in | arXiv.org |
---|---|
Main Authors | , , , |
Format | Paper |
Language | English |
Published |
Ithaca
Cornell University Library, arXiv.org
14.03.2023
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Rising concerns about privacy and anonymity preservation of deep learning models have facilitated research in data-free learning (DFL). For the first time, we identify that for data-scarce tasks like Sketch-Based Image Retrieval (SBIR), where the difficulty in acquiring paired photos and hand-drawn sketches limits data-dependent cross-modal learning algorithms, DFL can prove to be a much more practical paradigm. We thus propose Data-Free (DF)-SBIR, where, unlike existing DFL problems, pre-trained, single-modality classification models have to be leveraged to learn a cross-modal metric-space for retrieval without access to any training data. The widespread availability of pre-trained classification models, along with the difficulty in acquiring paired photo-sketch datasets for SBIR justify the practicality of this setting. We present a methodology for DF-SBIR, which can leverage knowledge from models independently trained to perform classification on photos and sketches. We evaluate our model on the Sketchy, TU-Berlin, and QuickDraw benchmarks, designing a variety of baselines based on state-of-the-art DFL literature, and observe that our method surpasses all of them by significant margins. Our method also achieves mAPs competitive with data-dependent approaches, all the while requiring no training data. Implementation is available at \url{https://github.com/abhrac/data-free-sbir}. |
---|---|
AbstractList | Rising concerns about privacy and anonymity preservation of deep learning models have facilitated research in data-free learning (DFL). For the first time, we identify that for data-scarce tasks like Sketch-Based Image Retrieval (SBIR), where the difficulty in acquiring paired photos and hand-drawn sketches limits data-dependent cross-modal learning algorithms, DFL can prove to be a much more practical paradigm. We thus propose Data-Free (DF)-SBIR, where, unlike existing DFL problems, pre-trained, single-modality classification models have to be leveraged to learn a cross-modal metric-space for retrieval without access to any training data. The widespread availability of pre-trained classification models, along with the difficulty in acquiring paired photo-sketch datasets for SBIR justify the practicality of this setting. We present a methodology for DF-SBIR, which can leverage knowledge from models independently trained to perform classification on photos and sketches. We evaluate our model on the Sketchy, TU-Berlin, and QuickDraw benchmarks, designing a variety of baselines based on state-of-the-art DFL literature, and observe that our method surpasses all of them by significant margins. Our method also achieves mAPs competitive with data-dependent approaches, all the while requiring no training data. Implementation is available at \url{https://github.com/abhrac/data-free-sbir}. |
Author | Dutta, Anjan Yi-Zhe, Song Bhunia, Ayan Kumar Chaudhuri, Abhra |
Author_xml | – sequence: 1 givenname: Abhra surname: Chaudhuri fullname: Chaudhuri, Abhra – sequence: 2 givenname: Ayan surname: Bhunia middlename: Kumar fullname: Bhunia, Ayan Kumar – sequence: 3 givenname: Song surname: Yi-Zhe fullname: Yi-Zhe, Song – sequence: 4 givenname: Anjan surname: Dutta fullname: Dutta, Anjan |
BookMark | eNrjYmDJy89LZWLgNDI2NtS1MDEy4mDgLS7OMjAwMDIzNzI1NeZkUHNJLEnUdStKTVUIzk4tSc7QdUosTk1R8MxNTE9VCEotKcpMLUvM4WFgTUvMKU7lhdLcDMpuriHOHroFRfmFpanFJfFZ-aVFeUCpeCNzC3MDQ1MDczNj4lQBAPHeMEc |
ContentType | Paper |
Copyright | 2023. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
Copyright_xml | – notice: 2023. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
DBID | 8FE 8FG ABJCF ABUWG AFKRA AZQEC BENPR BGLVJ CCPQU DWQXO HCIFZ L6V M7S PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
DatabaseName | ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central (Alumni) ProQuest Central ProQuest Central Essentials AUTh Library subscriptions: ProQuest Central Technology Collection ProQuest One Community College ProQuest Central SciTech Premium Collection (Proquest) (PQ_SDU_P3) ProQuest Engineering Collection ProQuest Engineering Database Publicly Available Content Database ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection |
DatabaseTitle | Publicly Available Content Database Engineering Database Technology Collection ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Technology Collection ProQuest SciTech Collection ProQuest Central China ProQuest Central ProQuest Engineering Collection ProQuest One Academic UKI Edition ProQuest Central Korea Materials Science & Engineering Collection ProQuest One Academic Engineering Collection |
DatabaseTitleList | Publicly Available Content Database |
Database_xml | – sequence: 1 dbid: 8FG name: ProQuest Technology Collection url: https://search.proquest.com/technologycollection1 sourceTypes: Aggregation Database |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Physics |
EISSN | 2331-8422 |
Genre | Working Paper/Pre-Print |
GroupedDBID | 8FE 8FG ABJCF ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS AZQEC BENPR BGLVJ CCPQU DWQXO FRJ HCIFZ L6V M7S M~E PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
ID | FETCH-proquest_journals_27870150763 |
IEDL.DBID | BENPR |
IngestDate | Tue Sep 24 23:35:04 EDT 2024 |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-proquest_journals_27870150763 |
OpenAccessLink | https://www.proquest.com/docview/2787015076/abstract/?pq-origsite=%requestingapplication% |
PQID | 2787015076 |
PQPubID | 2050157 |
ParticipantIDs | proquest_journals_2787015076 |
PublicationCentury | 2000 |
PublicationDate | 20230314 |
PublicationDateYYYYMMDD | 2023-03-14 |
PublicationDate_xml | – month: 03 year: 2023 text: 20230314 day: 14 |
PublicationDecade | 2020 |
PublicationPlace | Ithaca |
PublicationPlace_xml | – name: Ithaca |
PublicationTitle | arXiv.org |
PublicationYear | 2023 |
Publisher | Cornell University Library, arXiv.org |
Publisher_xml | – name: Cornell University Library, arXiv.org |
SSID | ssj0002672553 |
Score | 3.4507358 |
SecondaryResourceType | preprint |
Snippet | Rising concerns about privacy and anonymity preservation of deep learning models have facilitated research in data-free learning (DFL). For the first time, we... |
SourceID | proquest |
SourceType | Aggregation Database |
SubjectTerms | Algorithms Availability Classification Deep learning Image retrieval Machine learning Privacy Sketches Training |
Title | Data-Free Sketch-Based Image Retrieval |
URI | https://www.proquest.com/docview/2787015076/abstract/ |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1LS8NAEB7aBsFbfaG2loDibcluspvGk1BNjEJLqQq9lU12chFrTePV3-5OSPQg9LgsDMswM9-8lg_giniNTKRumCq4YTLgnGUcNRM6MhoLxFzQ3-HpLExf5dNSLTuQtn9haK2yjYl1oDYfOfXIPZ8si7KX0NMZdQHyyrvdfDLij6I5a0Om0QXHF5IGts4kns0Xv_0WPxzb7Dn4F3JrHEn64Mz1BssD6OD6EPbq9ct8ewTX97rSLCkR3ec3UiSbWHQx7uO7dXd3UbNeWZM4hsskfrlLWSt81ZjCdvX38OAEeramx1NwraOobBzIkAeFLVFMJLQyUqEuCh5Z3DqD4S5J57uvB7BPrOi0KiXkEHpV-YUXFjurbATdKHkYNcqxp-l3_APBrnig |
link.rule.ids | 786,790,12792,21416,33408,33779,43635,43840 |
linkProvider | ProQuest |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1LT8MwDLZgE2I3nuIxoBKIW0S6JF05IfEoHWwTgiHtVqWNc5kGoy3_nzjq4IC0cyQrcmx_tmPrA7ggXiMTq2umLDdMCs5ZzlGzUMdGo0UsQtodHo2j9F0-TdW0abhVzVjlMib6QG0-C-qRX_XIsih7iW4WX4xYo-h3taHQWIe2FA46aVM8efztsfSivsuYxb8w67Ej2YL2i15guQ1r-LEDG37ksqh24fJe15olJWLwNiPlsVuHKCYYzJ2LB6-e6cqZwR6cJw-Tu5QthWfN81fZ32XFPrRcHY8HEDjnUHlfyIgL68oSE4daGalQW8tjh1WH0F0l6Wj18RlsppPRMBsOxs_H0CFWdBqVCmUXWnX5jScOO-v81CvoB-_xdFY |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Data-Free+Sketch-Based+Image+Retrieval&rft.jtitle=arXiv.org&rft.au=Chaudhuri%2C+Abhra&rft.au=Bhunia%2C+Ayan+Kumar&rft.au=Yi-Zhe%2C+Song&rft.au=Dutta%2C+Anjan&rft.date=2023-03-14&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422 |