A fully-automatic image colorization scheme using improved CycleGAN with skip connections

• Published in: Multimedia tools and applications Vol. 80; no. 17; pp. 26465 - 26492
• Main Authors: Huang, Shanshan, Jin, Xin, Jiang, Qian, Li, Jie, Lee, Shin-Jye, Wang, Puming, Yao, Shaowen
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.07.2021; Springer Nature B.V
• Subjects: Color imagery; Colorization; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Generative adversarial networks; Gray scale; Multimedia Information Systems; Special Purpose and Application-Based Systems; Deep learning, Image colorization; Multimedia processing; Cycle-consistent adversarial network
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-021-10881-5

Image colorization is the process of assigning different RGB values to each pixel of a given grayscale image to obtain the corresponding colorized image. In this work, we propose a new automatic image colorization method based on the modified cycle-consistent generative adversarial network (CycleGAN). This method can generate a natural color image with only one given gray image without reference image or manual interaction. In the proposed method, we first modify the original network structure by combining a u-shaped network with a skip connection to improve the ability of feature representation in image colorization. Meanwhile, we design a compounded loss function to measure the errors between the ground-truth image and the predicted result to improve the authenticity and naturalness of the colorized image; further, we also add the detail loss function to ensure that the details of the generated color and grayscale images are substantially similar. Finally, the performance of the proposed model is verified on different datasets. Experiments show that our method can generate more realistic color images when compared to other methods.