Unifying Color and Texture Transfer for Predictive Appearance Manipulation

• Published in: Computer graphics forum Vol. 34; no. 4; pp. 53 - 63
• Main Authors: Okura, Fumio, Vanhoey, Kenneth, Bousseau, Adrien, Efros, Alexei A., Drettakis, George
• Format: Journal Article
• Language: English
• Published: Oxford Blackwell Publishing Ltd 01.07.2015; Wiley
• Subjects: Algorithms; and texture; Categories and Subject Descriptors (according to ACM CCS); Color; Computer graphics; Computer Science; Computer Vision and Pattern Recognition; Customization; I.3.3 [Computer Graphics]: Picture/Image Generation; I.3.7 [Computer Graphics]: Color; I.3.7 [Computer Graphics]: Color, shading, shadowing, and texture; Leaves; Mathematical models; shading; shadowing; Studies; Surface layer; Texture; Trees; United States > US
• ISSN: 0167-7055; 1467-8659
• DOI: 10.1111/cgf.12678

Recent color transfer methods use local information to learn the transformation from a source to an exemplar image, and then transfer this appearance change to a target image. These solutions achieve very successful results for general mood changes, e.g., changing the appearance of an image from “sunny” to “overcast”. However, such methods have a hard time creating new image content, such as leaves on a bare tree. Texture transfer, on the other hand, can synthesize such content but tends to destroy image structure. We propose the first algorithm that unifies color and texture transfer, outperforming both by leveraging their respective strengths. A key novelty in our approach resides in teasing apart appearance changes that can be modeled simply as changes in color versus those that require new image content to be generated. Our method starts with an analysis phase which evaluates the success of color transfer by comparing the exemplar with the source. This analysis then drives a selective, iterative texture transfer algorithm that simultaneously predicts the success of color transfer on the target and synthesizes new content where needed. We demonstrate our unified algorithm by transferring large temporal changes between photographs, such as change of season – e.g., leaves on bare trees or piles of snow on a street – and flooding.