Lighting transfer across multiple views through local color transforms

• Published in: Computational visual media (Beijing) Vol. 3; no. 4; pp. 315 - 324
• Main Authors: Zhang, Qian, Laffont, Pierre-Yves, Sim, Terence
• Format: Journal Article
• Language: English
• Published: Beijing Tsinghua University Press 01.12.2017; Springer Nature B.V; SpringerOpen
• Subjects: Artificial Intelligence; Browsing; Color; Computer Graphics; Computer Science; Illumination; Image Editing; Image enhancement; Image Processing and Computer Vision; Lighting; Photo Collection; Relighting; Research Article; Time-Lapse; User Interfaces and Human Computer Interaction; Relighting; Image Editing; Photo Collection; Time-Lapse
• ISSN: 2096-0433; 2096-0662
• DOI: 10.1007/s41095-017-0085-5

We present a method for transferring lighting between photographs of a static scene. Our method takes as input a photo collection depicting a scene with varying viewpoints and lighting conditions. We cast lighting transfer as an edit propagation problem, where the transfer of local illumination across images is guided by sparse correspondences obtained through multi-view stereo. Instead of directly propagating color, we learn local color transforms from corresponding patches in pairs of images and propagate these transforms in an edge-aware manner to regions with no correspondences. Our color transforms model the large variability of appearance changes in local regions of the scene, and are robust to missing or inaccurate correspondences. The method is fully automatic and can transfer strong shadows between images. We show applications of our image relighting method for enhancing photographs, browsing photo collections with harmonized lighting, and generating synthetic time-lapse sequences.