Learning to Separate Multiple Illuminants in a Single Image

• Published in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 3775 - 3784
• Main Authors: Hui, Zhuo, Chakrabarti, Ayan, Sunkavalli, Kalyan, Sankaranarayanan, Aswin C.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2019
• Subjects: Computational Photography; Deep Learning; Low-level Vision
• ISSN: 2575-7075
• DOI: 10.1109/CVPR.2019.00390

We present a method to separate a single image captured under two illuminants, with different spectra, into the two images corresponding to the appearance of the scene under each individual illuminant. We do this by training a deep neural network to predict the per-pixel reflectance chromaticity of the scene, which we use in conjunction with a previous flash/no-flash image-based separation algorithm to produce the final two output images. We design our reflectance chromaticity network and loss functions by incorporating intuitions from the physics of image formation. We show that this leads to significantly better performance than other single image techniques and even approaches the quality of the two image separation method.