MuralDiff: Diffusion for Ancient Murals Restoration on Large-Scale Pre-Training

• Published in: IEEE transactions on emerging topics in computational intelligence Vol. 8; no. 3; pp. 2169 - 2181
• Main Authors: Xu, Zishan, Zhang, Xiaofeng, Chen, Wei, Liu, Jueting, Xu, Tingting, Wang, Zehua
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.06.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation models; crack detection; Cultural differences; Cultural heritage; Cultural resources; diffusion model; Generative adversarial networks; Image edge detection; image inpainting; Image restoration; large-scale pretraining; Mural restoration; Murals; Restoration; Task analysis; Training
• ISSN: 2471-285X; 2471-285X
• DOI: 10.1109/TETCI.2024.3359038

This paper focuses on the crack detection and digital restoration of ancient mural cultural heritage, proposing a comprehensive method that combines the Unet network structure and diffusion model. Firstly, the Unet network structure is used for efficient crack detection in murals by constructing an ancient mural image dataset for training and validation, achieving accurate identification of mural cracks. Next, an edge-guided optimized masking strategy is adopted for mural restoration, effectively preserving the information of the murals and reducing the damage to the original murals during the restoration process. Lastly, a diffusion model is employed for digital restoration of murals, improving the restoration performance by adjusting parameters to achieve natural repair of mural cracks. Experimental results show that comprehensive method based on the Unet network and diffusion model has significant advantages in the tasks of crack detection and digital restoration of murals, providing a novel and effective approach for the protection and restoration of ancient murals. In addition, this research has significant implications for the technological development in the field of mural restoration and cultural heritage preservation, contributing to the advancement and technological innovation in related fields.