Separating Shading and Reflectance From Cartoon Illustrations

• Published in: IEEE transactions on visualization and computer graphics Vol. 30; no. 7; pp. 3664 - 3679
• Main Authors: Ma, Ziheng, Li, Chengze, Liu, Xueting, Wu, Huisi, Wen, Zhenkun
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.07.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artists; Cartoon; Cartoons; Computer graphics; deep learning; Estimation; Image color analysis; layer decomposition; Lighting; Reflectivity; Shading; shading extraction; shading removal; Smoothing methods; Task analysis; Visualization
• ISSN: 1077-2626; 1941-0506; 1941-0506
• DOI: 10.1109/TVCG.2023.3239364

Shading plays an important role in cartoon drawings to present the 3D lighting and depth information in a 2D image to improve the visual information and pleasantness. But it also introduces apparent challenges in analyzing and processing the cartoon drawings for different computer graphics and vision applications, such as segmentation, depth estimation, and relighting. Extensive research has been made in removing or separating the shading information to facilitate these applications. Unfortunately, the existing researches only focused on natural images, which are natively different from cartoons since the shading in natural images is physically correct and can be modeled based on physical priors. However, shading in cartoons is manually created by artists, which may be imprecise, abstract, and stylized. This makes it extremely difficult to model the shading in cartoon drawings. Without modeling the shading prior, in the paper, we propose a learning-based solution to separate the shading from the original colors using a two-branch system consisting of two subnetworks. To the best of our knowledge, our method is the first attempt in separating shading information from cartoon drawings. Our method significantly outperforms the methods tailored for natural images. Extensive evaluations have been performed with convincing results in all cases.