High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network

• Published in: Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 9387 - 9395
• Main Authors: Liang, Jie, Zeng, Hui, Zhang, Lei
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.01.2021
• Subjects: Graphics processing units; Image color analysis; Image resolution; Laplace equations; Lighting; Real-time systems; Visualization
• ISSN: 1063-6919
• DOI: 10.1109/CVPR46437.2021.00927

Existing image-to-image translation (I2IT) methods are either constrained to low-resolution images or long inference time due to their heavy computational burden on the convolution of high-resolution feature maps. In this paper, we focus on speeding-up the high-resolution photorealistic I2IT tasks based on closed-form Laplacian pyramid decomposition and reconstruction. Specifically, we reveal that the attribute transformations, such as illumination and color manipulation, relate more to the low-frequency component, while the content details can be adaptively refined on high-frequency components. We consequently propose a Laplacian Pyramid Translation Network (LPTN) to simultaneously perform these two tasks, where we design a lightweight network for translating the low-frequency component with reduced resolution and a progressive masking strategy to efficiently refine the high-frequency ones. Our model avoids most of the heavy computation consumed by processing high-resolution feature maps and faithfully preserves the image details. Extensive experimental results on various tasks demonstrate that the proposed method can translate 4K images in real-time using one normal GPU while achieving comparable transformation performance against existing methods. Datasets and codes are available: https://github.com/csjliang/LPTN.