Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Erasers

Concept erasure in text-to-image diffusion models aims to disable pre-trained diffusion models from generating images related to a target concept. To perform reliable concept erasure, the properties of robustness and locality are desirable. The former refrains the model from producing images associa...

Full description

Saved in:
Bibliographic Details
Main Authors Huang, Chi-Pin, Chang, Kai-Po, Tsai, Chung-Ting, Lai, Yung-Hsuan, Yang, Fu-En, Wang, Yu-Chiang Frank
Format Journal Article
LanguageEnglish
Published 29.11.2023
Subjects
Computer Science - Computer Vision and Pattern Recognition
Computer Science - Learning
Online AccessGet full text

Cover

More Information
Summary:Concept erasure in text-to-image diffusion models aims to disable pre-trained diffusion models from generating images related to a target concept. To perform reliable concept erasure, the properties of robustness and locality are desirable. The former refrains the model from producing images associated with the target concept for any paraphrased or learned prompts, while the latter preserves its ability in generating images with non-target concepts. In this paper, we propose Reliable Concept Erasing via Lightweight Erasers (Receler). It learns a lightweight Eraser to perform concept erasing while satisfying the above desirable properties through the proposed concept-localized regularization and adversarial prompt learning scheme. Experiments with various concepts verify the superiority of Receler over previous methods.
DOI:10.48550/arxiv.2311.17717