Automatic Colorization of Anime Style Illustrations Using a Two-Stage Generator

• Published in: Applied sciences Vol. 10; no. 23; p. 8699
• Main Authors: Lee, Yeongseop, Lee, Seongjin
• Format: Journal Article
• Language: English
• Published: MDPI AG 01.12.2020
• Subjects: automatic colorization; GAN; histogram equalization; line detection mode; line-art colorization; loss function
• ISSN: 2076-3417; 2076-3417
• DOI: 10.3390/app10238699

Line-arts are used in many ways in the media industry. However, line-art colorization is tedious, labor-intensive, and time consuming. For such reasons, a Generative Adversarial Network (GAN)-based image-to-image colorization method has received much attention because of its promising results. In this paper, we propose to use color a point hinting method with two GAN-based generators used for enhancing the image quality. To improve the coloring performance of drawing with various line styles, generator takes account of the loss of the line-art. We propose a Line Detection Model (LDM) which is used in measuring line loss. LDM is a method of extracting line from a color image. We also propose histogram equalizer in the input line-art to generalize the distribution of line styles. This approach allows the generalization of the distribution of line style without increasing the complexity of inference stage. In addition, we propose seven segment hint pointing constraints to evaluate the colorization performance of the model with Fréchet Inception Distance (FID) score. We present visual and qualitative evaluations of the proposed methods. The result shows that using histogram equalization and LDM enabled line loss exhibits the best result. The Base model with XDoG (eXtended Difference-Of-Gaussians)generated line-art with and without color hints exhibits FID for colorized images score of 35.83 and 44.70, respectively, whereas the proposed model in the same scenario exhibits 32.16 and 39.77, respectively.