UPHDR-GAN: Generative Adversarial Network for High Dynamic Range Imaging With Unpaired Data

• Published in: IEEE transactions on circuits and systems for video technology Vol. 32; no. 11; pp. 7532 - 7546
• Main Authors: Li, Ru, Wang, Chuan, Wang, Jue, Liu, Guanghui, Zhang, Heng-Yu, Zeng, Bing, Liu, Shuaicheng
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.11.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; Datasets; Domains; Dynamic range; Dynamics; Fuses; generative adversarial network; Generative adversarial networks; Image processing; Image quality; Image reconstruction; Machine learning; Multi-exposure HDR imaging; Optimization; Training; unpaired data
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2022.3190057

The paper proposes a method to effectively fuse multi-exposure inputs and generate high-quality high dynamic range (HDR) images with unpaired datasets. Deep learning-based HDR image generation methods rely heavily on paired datasets. The ground truth images play a leading role in generating reasonable HDR images. Datasets without ground truth are hard to be applied to train deep neural networks. Recently, Generative Adversarial Networks (GAN) have demonstrated their potentials of translating images from source domain <inline-formula> <tex-math notation="LaTeX">X </tex-math></inline-formula> to target domain <inline-formula> <tex-math notation="LaTeX">Y </tex-math></inline-formula> in the absence of paired examples. In this paper, we propose a GAN-based network for solving such problems while generating enjoyable HDR results, named UPHDR-GAN. The proposed method relaxes the constraint of the paired dataset and learns the mapping from the LDR domain to the HDR domain. Although the pair data are missing, UPHDR-GAN can properly handle the ghosting artifacts caused by moving objects or misalignments with the help of the modified GAN loss, the improved discriminator network and the useful initialization phase. The proposed method preserves the details of important regions and improves the total image perceptual quality. Qualitative and quantitative comparisons against the representative methods demonstrate the superiority of the proposed UPHDR-GAN.