Content-Aware Image Color Editing with Auxiliary Color Restoration Tasks

• Published in: 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) pp. 5180 - 5189
• Main Authors: Ren, Yixuan, Shi, Jing, Zhang, Zhifei, Fan, Yifei, Lin, Zhe, He, Bo, Shrivastava, Abhinav
• Format: Conference Proceeding
• Language: English
• Published: IEEE 03.01.2024
• Subjects: Adaptation models; Algorithms; Computational modeling; Computational photography; etc; Generative models for image; Harmonic analysis; image and video synthesis; Image color analysis; Measurement; Semantics; Training; video
• ISSN: 2642-9381
• DOI: 10.1109/WACV57701.2024.00511

Diversified image color editing is typically modeled as a multimodal image-to-image translation (MMI2IT) problem with an impact on multiple applications such as photo enhancement and retouching. Although previous GAN-based algorithms successfully generate diverse edits with clear control, we observe two issues remaining: Firstly, they tend to apply the same color style to all kinds of input images when sampling with the same style latent, regardless of the input content and scenes. Secondly, they usually edit the color style globally in an image and fail to keep each semantic region and instance in harmonic colors individually. We attribute these issues to the strong independence between the style latent and the condition image in most current MMI2IT methods.To edit the raw image into a more harmonic direction with awareness of its global content and local semantics, we introduce auxiliary color restoration tasks by reducing the input color information and training jointly. We also increase the model's capacity and enrich the noise's locality with diffusion models. Furthermore, we propose a new set of metrics to measure the content-awareness of MMI2IT models, that is, how the generated style is adaptive to the input image's content. Our model is also extensible to several downstream applications including exemplar-based color editing and language-guided color editing, without imposing extra demands on the already trained model.