Photorealistic Style Transfer via Wavelet Transforms

Recent style transfer models have provided promising artistic results. However, given a photograph as a reference style, existing methods are limited by spatial distortions or unrealistic artifacts, which should not happen in real photographs. We introduce a theoretically sound correction to the net...

Full description

Saved in:
Bibliographic Details
Published in2019 IEEE/CVF International Conference on Computer Vision (ICCV) pp. 9035 - 9044
Main Authors Yoo, Jaejun, Uh, Youngjung, Chun, Sanghyuk, Kang, Byeongkyu, Ha, Jung-Woo
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.10.2019
Subjects
Decoding
Distortion
Image reconstruction
Image resolution
Network architecture
Wavelet transforms
Online AccessGet full text

Cover

More Information
Summary:Recent style transfer models have provided promising artistic results. However, given a photograph as a reference style, existing methods are limited by spatial distortions or unrealistic artifacts, which should not happen in real photographs. We introduce a theoretically sound correction to the network architecture that remarkably enhances photorealism and faithfully transfers the style. The key ingredient of our method is wavelet transforms that naturally fits in deep networks. We propose a wavelet corrected transfer based on whitening and coloring transforms (WCT2) that allows features to preserve their structural information and statistical properties of VGG feature space during stylization. This is the first and the only end-to-end model that can stylize a 1024x1024 resolution image in 4.7 seconds, giving a pleasing and photorealistic quality without any post-processing. Last but not least, our model provides a stable video stylization without temporal constraints. Our code, generated images, pre-trained models and supplementary documents are all available at https://github.com/ClovaAI/WCT2.
ISSN:2380-7504
DOI:10.1109/ICCV.2019.00913