Generative Edge Detection with Stable Diffusion
Edge detection is typically viewed as a pixel-level classification problem mainly addressed by discriminative methods. Recently, generative edge detection methods, especially diffusion model based solutions, are initialized in the edge detection task. Despite great potential, the retraining of task-...
Saved in:
Published in | arXiv.org |
---|---|
Main Authors | , , , , , |
Format | Paper |
Language | English |
Published |
Ithaca
Cornell University Library, arXiv.org
04.10.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Edge detection is typically viewed as a pixel-level classification problem mainly addressed by discriminative methods. Recently, generative edge detection methods, especially diffusion model based solutions, are initialized in the edge detection task. Despite great potential, the retraining of task-specific designed modules and multi-step denoising inference limits their broader applications. Upon closer investigation, we speculate that part of the reason is the under-exploration of the rich discriminative information encoded in extensively pre-trained large models (\eg, stable diffusion models). Thus motivated, we propose a novel approach, named Generative Edge Detector (GED), by fully utilizing the potential of the pre-trained stable diffusion model. Our model can be trained and inferred efficiently without specific network design due to the rich high-level and low-level prior knowledge empowered by the pre-trained stable diffusion. Specifically, we propose to finetune the denoising U-Net and predict latent edge maps directly, by taking the latent image feature maps as input. Additionally, due to the subjectivity and ambiguity of the edges, we also incorporate the granularity of the edges into the denoising U-Net model as one of the conditions to achieve controllable and diverse predictions. Furthermore, we devise a granularity regularization to ensure the relative granularity relationship of the multiple predictions. We conduct extensive experiments on multiple datasets and achieve competitive performance (\eg, 0.870 and 0.880 in terms of ODS and OIS on the BSDS test dataset). |
---|---|
AbstractList | Edge detection is typically viewed as a pixel-level classification problem mainly addressed by discriminative methods. Recently, generative edge detection methods, especially diffusion model based solutions, are initialized in the edge detection task. Despite great potential, the retraining of task-specific designed modules and multi-step denoising inference limits their broader applications. Upon closer investigation, we speculate that part of the reason is the under-exploration of the rich discriminative information encoded in extensively pre-trained large models (\eg, stable diffusion models). Thus motivated, we propose a novel approach, named Generative Edge Detector (GED), by fully utilizing the potential of the pre-trained stable diffusion model. Our model can be trained and inferred efficiently without specific network design due to the rich high-level and low-level prior knowledge empowered by the pre-trained stable diffusion. Specifically, we propose to finetune the denoising U-Net and predict latent edge maps directly, by taking the latent image feature maps as input. Additionally, due to the subjectivity and ambiguity of the edges, we also incorporate the granularity of the edges into the denoising U-Net model as one of the conditions to achieve controllable and diverse predictions. Furthermore, we devise a granularity regularization to ensure the relative granularity relationship of the multiple predictions. We conduct extensive experiments on multiple datasets and achieve competitive performance (\eg, 0.870 and 0.880 in terms of ODS and OIS on the BSDS test dataset). |
Author | Ren, Jiahui Ling, Haibin Zhang, Jing Huang, Yaping Mochu Xiang Zhou, Caixia |
Author_xml | – sequence: 1 givenname: Caixia surname: Zhou fullname: Zhou, Caixia – sequence: 2 givenname: Yaping surname: Huang fullname: Huang, Yaping – sequence: 3 fullname: Mochu Xiang – sequence: 4 givenname: Jiahui surname: Ren fullname: Ren, Jiahui – sequence: 5 givenname: Haibin surname: Ling fullname: Ling, Haibin – sequence: 6 givenname: Jing surname: Zhang fullname: Zhang, Jing |
BookMark | eNqNit0OwTAYQBshMewdmrhe9G821wz33C_FV7osLe1XXt8uPICrk5xzZmTsvIMRyYSUvKiVEFOSx9gxxsS6EmUpM7I6gIOg0b6BNrc70B0gXNF6Rz8WH_SE-tIP1hqT4mAXZGJ0HyH_cU6W--a8PRbP4F8JIradT8ENqZWcy1pVim3kf9cXQbQ0Pg |
ContentType | Paper |
Copyright | 2024. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
Copyright_xml | – notice: 2024. This work is published under http://arxiv.org/licenses/nonexclusive-distrib/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
DBID | 8FE 8FG ABJCF ABUWG AFKRA AZQEC BENPR BGLVJ CCPQU DWQXO HCIFZ L6V M7S PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
DatabaseName | ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central (Alumni) ProQuest Central UK/Ireland ProQuest Central Essentials ProQuest Central Technology Collection ProQuest One Community College ProQuest Central Korea SciTech Premium Collection ProQuest Engineering Collection Engineering Database Publicly Available Content Database ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection |
DatabaseTitle | Publicly Available Content Database Engineering Database Technology Collection ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Technology Collection ProQuest SciTech Collection ProQuest Central China ProQuest Central ProQuest Engineering Collection ProQuest One Academic UKI Edition ProQuest Central Korea Materials Science & Engineering Collection ProQuest One Academic Engineering Collection |
DatabaseTitleList | Publicly Available Content Database |
Database_xml | – sequence: 1 dbid: 8FG name: ProQuest Technology Collection url: https://search.proquest.com/technologycollection1 sourceTypes: Aggregation Database |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Physics |
EISSN | 2331-8422 |
Genre | Working Paper/Pre-Print |
GroupedDBID | 8FE 8FG ABJCF ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS AZQEC BENPR BGLVJ CCPQU DWQXO FRJ HCIFZ L6V M7S M~E PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
ID | FETCH-proquest_journals_31138474093 |
IEDL.DBID | BENPR |
IngestDate | Thu Oct 10 20:51:21 EDT 2024 |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-proquest_journals_31138474093 |
OpenAccessLink | https://www.proquest.com/docview/3113847409?pq-origsite=%requestingapplication% |
PQID | 3113847409 |
PQPubID | 2050157 |
ParticipantIDs | proquest_journals_3113847409 |
PublicationCentury | 2000 |
PublicationDate | 20241004 |
PublicationDateYYYYMMDD | 2024-10-04 |
PublicationDate_xml | – month: 10 year: 2024 text: 20241004 day: 04 |
PublicationDecade | 2020 |
PublicationPlace | Ithaca |
PublicationPlace_xml | – name: Ithaca |
PublicationTitle | arXiv.org |
PublicationYear | 2024 |
Publisher | Cornell University Library, arXiv.org |
Publisher_xml | – name: Cornell University Library, arXiv.org |
SSID | ssj0002672553 |
Score | 3.5746644 |
SecondaryResourceType | preprint |
Snippet | Edge detection is typically viewed as a pixel-level classification problem mainly addressed by discriminative methods. Recently, generative edge detection... |
SourceID | proquest |
SourceType | Aggregation Database |
SubjectTerms | Controllability Datasets Edge detection Feature maps Network design Noise reduction Regularization |
Title | Generative Edge Detection with Stable Diffusion |
URI | https://www.proquest.com/docview/3113847409 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwY2BQMTVIMwW2UpN0QafL6ZoAuxC6lmnAjJeamJRqkmqUaGySBhrv8PUz8wg18YowjYAOuBVDl1XCykRwQZ2SnwwaI9c3NjQ0BpakwO6IfUGhLujWKNDsKvQKDWYGViNDE9A0LauTq19AEHyUxcjMHNhmNsYoaMG1h5sgA2tAYkFqkRADU2qeMAM7eNFlcrEIgz7kzGdQgaPgmpKequCSWgJeGZWnABoeVQA2BJNygKKZaWmloEEtUQZlN9cQZw9dmC3x0JRQHI9wt7EYAwuwS58qwaCQlGhimJKYagbeR2qRbG6RappmapiYnJSYDCy4Uo0kGWTwmSSFX1qagcsIWPWCl5yZyDCwlBSVpsoCq86SJDkGZgs3dzloKAF5vnWuAPv2eEY |
link.rule.ids | 786,790,12792,21416,33406,33777,43633,43838 |
linkProvider | ProQuest |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV3fS8MwED60RfTNn_hjakBfy2ybdvVJUDuqbmXIhL2VJL3IQOZcu__fS8z0QdhrAkkIyXd3X77cAVwnNzohL1UGJrtcwCmECG41XTwUEjlGIuba8B3DMi3e-PMkmTjCrXGyyhUmWqCuP5XhyLtxGMaEpBSO3M2_AlM1yryuuhIam-CblJuZB_59Xo5ef1mWKO2Rzxz_A1prPfq74I_EHBd7sIGzfdiyokvVHED3J-ezARyW1-_IHrG1yqgZM_QoI0dQflDrVOulIbUO4aqfjx-KYDVL5U5CU_2tOz4Cj0J6PAYmBQ9rgan9R5qpXoaJTkKhpFAEXBidQGfdSKfruy9huxgPB9XgqXw5g52IzLCVn_EOeO1iiedkRlt54fbqGyHseSU |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Generative+Edge+Detection+with+Stable+Diffusion&rft.jtitle=arXiv.org&rft.au=Zhou%2C+Caixia&rft.au=Huang%2C+Yaping&rft.au=Mochu+Xiang&rft.au=Ren%2C+Jiahui&rft.date=2024-10-04&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422 |