Generative Edge Detection with Stable Diffusion
Edge detection is typically viewed as a pixel-level classification problem mainly addressed by discriminative methods. Recently, generative edge detection methods, especially diffusion model based solutions, are initialized in the edge detection task. Despite great potential, the retraining of task-...
Saved in:
Main Authors | , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
03.10.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Edge detection is typically viewed as a pixel-level classification problem
mainly addressed by discriminative methods. Recently, generative edge detection
methods, especially diffusion model based solutions, are initialized in the
edge detection task. Despite great potential, the retraining of task-specific
designed modules and multi-step denoising inference limits their broader
applications. Upon closer investigation, we speculate that part of the reason
is the under-exploration of the rich discriminative information encoded in
extensively pre-trained large models (\eg, stable diffusion models). Thus
motivated, we propose a novel approach, named Generative Edge Detector (GED),
by fully utilizing the potential of the pre-trained stable diffusion model. Our
model can be trained and inferred efficiently without specific network design
due to the rich high-level and low-level prior knowledge empowered by the
pre-trained stable diffusion. Specifically, we propose to finetune the
denoising U-Net and predict latent edge maps directly, by taking the latent
image feature maps as input. Additionally, due to the subjectivity and
ambiguity of the edges, we also incorporate the granularity of the edges into
the denoising U-Net model as one of the conditions to achieve controllable and
diverse predictions. Furthermore, we devise a granularity regularization to
ensure the relative granularity relationship of the multiple predictions. We
conduct extensive experiments on multiple datasets and achieve competitive
performance (\eg, 0.870 and 0.880 in terms of ODS and OIS on the BSDS test
dataset). |
---|---|
DOI: | 10.48550/arxiv.2410.03080 |