CLPFusion: A Latent Diffusion Model Framework for Realistic Chinese Landscape Painting Style Transfer

• Published in: Computer animation and virtual worlds Vol. 36; no. 3
• Main Authors: Pan, Jiahui, Li, Frederick W. B., Yang, Bailin, Nan, Fangzhe
• Format: Journal Article
• Language: English
• Published: Hoboken, USA John Wiley & Sons, Inc 01.05.2025; Wiley Subscription Services, Inc
• Subjects: Artificial neural networks; Chinese landscape painting; Diffusion models; Generative adversarial networks; Image processing; Landscape art; State space models; style transfer
• DOI: 10.1002/cav.70053
• ISSN: 1546-4261

ABSTRACT This study focuses on transforming real‐world scenery into Chinese landscape painting masterpieces through style transfer. Traditional methods using convolutional neural networks (CNNs) and generative adversarial networks (GANs) often yield inconsistent patterns and artifacts. The rise of diffusion models (DMs) presents new opportunities for realistic image generation, but their inherent noise characteristics make it challenging to synthesize pure white or black images. Consequently, existing DM‐based methods struggle to capture the unique style and color information of Chinese landscape paintings. To overcome these limitations, we propose CLPFusion, a novel framework that leverages pre‐trained diffusion models for artistic style transfer. A key innovation is the Bidirectional State Space Models‐CrossAttention (BiSSM‐CA) module, which efficiently learns and retains the distinct styles of Chinese landscape paintings. Additionally, we introduce two latent space feature adjustment methods, Latent‐AdaIN and Latent‐WCT, to enhance style modulation during inference. Experiments demonstrate that CLPFusion produces more realistic and artistic Chinese landscape paintings than existing approaches, showcasing its effectiveness and uniqueness in the field. We propose CLPFusion, a diffusion‐based artistic style transfer framework that integrates Bidirectional State Space Models‐CrossAttention (BiSSM‐CA) with latent space feature adjustment (Latent‐AdaIN and Latent‐WCT). Our method effectively enhances style retention and color fidelity for Chinese landscape painting generation.