BacklitNet: A dataset and network for backlit image enhancement

• Published in: Computer vision and image understanding Vol. 218; p. 103403
• Main Authors: Lv, Xiaoqian, Zhang, Shengping, Liu, Qinglin, Xie, Haozhe, Zhong, Bineng, Zhou, Huiyu
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01.04.2022
• Subjects: Backlit image enhancement; Deep learning; Image processing; Deep learning; Image processing; 41A10; 65D05; 65D17; Backlit image enhancement; 41A05
• ISSN: 1077-3142; 1090-235X
• DOI: 10.1016/j.cviu.2022.103403

Backlit images are usually taken when the light source is opposite to the camera. The uneven exposure (e.g., underexposure on the foreground and overexposure on the background) makes the backlit images more challenging than general image enhancement tasks that only need to increase or decrease the exposure on the whole images. Compared to traditional approaches, Convolutional Neural Networks perform well in enhancing images due to the abilities of exploiting contextual features. However, the lack of large benchmark datasets and specially designed models impedes the development of backlit image enhancement. In this paper, we build the first large-scale BAcklit Image Dataset (BAID), which contains 3000 backlit images and the corresponding ground truth manually adjusted by trained photographers. It covers a broad range of categories under different backlit conditions in both indoor and outdoor scenes. Furthermore, we propose a saliency guided backlit image enhancement network, namely BacklitNet, for robust and natural restoration of backlit images. In particular, our model innovatively combines a nested U-structure with bilateral grids, which enables fully extracting multi-scale saliency information and rapidly enhancing arbitrary resolution images. Moreover, a carefully designed loss function based on prior knowledge of brightness distribution of backlit images is proposed to enforce the network to focus more on backlit regions during the training phase. We evaluate the proposed method on the BAID dataset and two public small-scale backlit image datasets. Experimental results demonstrate that our method performs favorably against the state-of-the-art approaches. •We build the first large-scale backlit image dataset (BAID) which contains 3000 backlit images and the corresponding high-quality ground truth. The constructed dataset makes end-to-end learning of robust backlit enhancers possible and promotes the application of neural networks in backlit image enhancement.•We design a novel backlit image enhancement framework, which efficiently restores ill-exposed regions of backlit images based on subject salience and brightness distribution prior knowledge without damaging the well-exposed regions.•We evaluate the proposed method on the BAID dataset and two public small-scale backlit image datasets. The results show that our model achieves state-of-the-art performance both on visual effects and commonly-used metrics.