Progressive Dunhuang Murals Inpainting Based on Attention Mechanism

• Published in: 2023 IEEE 4th International Conference on Pattern Recognition and Machine Learning (PRML) pp. 350 - 356
• Main Authors: Li, Jiacheng, Shi, Yuqing, Liu, Wenjie, Wang, Jianhua, Du, Shiqiang
• Format: Conference Proceeding
• Language: English
• Published: IEEE 04.08.2023
• Subjects: Attention mechanism; Color; Dunhuang murals; Feature fusion; Image color analysis; Image edge detection; Image Inpainting; Machine learning; Merging; Semantic segmentation; Visualization
• DOI: 10.1109/PRML59573.2023.10348292

In this paper, we propose a novel approach for inpainting damaged areas of Dunhuang murals based on Recurrent Feature Reasoning Network (RFR-Net) with several innovative contributions. The Dunhuang murals are a precious cultural heritage that requires careful restoration and conservation. To address this challenge, we introduce an Efficient Attention (EA) mechanism on shallow features to capture more detailed information of the murals. Moreover, we propose a Attentional Residual Merging module (ARM) to merge intermediate feature maps for more effective feature fusion. And two new loss functions, HSV loss based on the HSV color space and edge loss, are introduced to improve color consistency and preserve the edges of murala during the inpainting process. Experimental results on a dataset of Dunhuang murals demonstrate that our proposed method achieves state-of-the-art performance in terms of both quantitative and qualitative evaluation. Our method effectively restores the missing areas of the murals while maintaining the color consistency and preserving the edges. The proposed ARM module effectively integrates intermediate features from different recursions, improving the accuracy of the inpainting results. The HSV loss function ensures that the inpainted areas match the surrounding colors of the mural, while the edge loss preserves the edges of the murals, which is crucial for maintaining the integrity of the artwork. Experimental results, both qualitatively and quantitatively, indicate that our proposed method achieves competitive outcomes compared to the contrastive algorithms.