COLA-Net: Collaborative Attention Network for Image Restoration

• Published in: IEEE transactions on multimedia Vol. 24; pp. 1366 - 1377
• Main Authors: Mou, Chong, Zhang, Jian, Fan, Xiaopeng, Liu, Hangfan, Wang, Ronggang
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Collaboration; Complexity; Computer architecture; Deep neural network; Degeneration; feature fusion; Finite element analysis; Image compression; Image denoising; Image restoration; Noise reduction; non-local attention; Self-similarity; Signal to noise ratio; Source code; Task analysis; Training; Visual perception; Visual signals
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2021.3063916

Local and non-local attention-based methods have been well studied in various image restoration tasks while leading to promising performance. However, most of the existing methods solely focus on one type of attention mechanism (local or non-local). Furthermore, by exploiting the self-similarity of natural images, existing pixel-wise non-local attention operations tend to give rise to deviations in the process of characterizing long-range dependence due to image degeneration. To overcome these problems, in this paper we propose a novel collaborative attention network (COLA-Net) for image restoration, as the first attempt to combine local and non-local attention mechanisms to restore image content in the areas with complex textures and with highly repetitive details respectively. In addition, an effective and robust patch-wise non-local attention model is developed to capture long-range feature correspondences through 3D patches. Extensive experiments on synthetic image denoising, real image denoising and compression artifact reduction tasks demonstrate that our proposed COLA-Net is able to achieve state-of-the-art performance in both peak signal-to-noise ratio and visual perception, while maintaining an attractive computational complexity. The source code is available on https://github.com/MC-E/COLA-Net .