FaceChain-SuDe: Building Derived Class to Inherit Category Attributes for One-Shot Subject-Driven Generation

• Published in: Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 7215 - 7224
• Main Authors: Qiao, Pengchong, Shang, Lei, Liu, Chang, Sun, Baigui, Ji, Xiangyang, Chen, Jie
• Format: Conference Proceeding
• Language: English
• Published: IEEE 16.06.2024
• Subjects: Buildings; Codes; Computer vision; derived class; Object oriented modeling; Pattern recognition; Semantics; subject-driven generation; Text to image
• ISSN: 1063-6919
• DOI: 10.1109/CVPR52733.2024.00689

Recently, subject-driven generation has garnered significant interest due to its ability to personalize text-to-image generation. Typical works focus on learning the new subject's private attributes. However, an important fact has not been taken seriously that a subject is not an isolated new concept but should be a specialization of a certain category in the pre-trained model. This results in the subject failing to comprehensively inherit the attributes in its category, causing poor attribute-related generations. In this paper, motivated by object-oriented programming, we model the subject as a derived class whose base class is its semantic category. This modeling enables the subject to inherit public attributes from its category while learning its private attributes from the user-provided example. Specifically, we propose a plug-and-play method, Subject-Derived regularization (SuDe). It constructs the base-derived class modeling by constraining the subject-driven generated images to semantically belong to the subject's category. Extensive experiments under three baselines and two backbones on various subjects show that our SuDe enables imaginative attribute-related generations while maintaining subject fidelity. For the codes, please refer to FaceChain.