Near-Infrared Fusion via Color Regularization for Haze and Color Distortion Removals

• Published in: IEEE transactions on circuits and systems for video technology Vol. 28; no. 11; pp. 3111 - 3126
• Main Authors: Son, Chang-Hwan, Zhang, Xiao-Ping
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.11.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Atmospheric modeling; Color; color correction; Coloring; Degradation; depth; Distortion; Electronic mail; Glucose; Haze; haze removal; Image color analysis; Imaging; Infrared imagery; Infrared imaging; multi-modal fusion; near-infrared imaging; Regularization; Visibility
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2017.2748150

Different from conventional haze removal methods based on a single image, near-infrared imaging can provide two types of multimodal images: one is the near-infrared image and the other is the visible color image. These two images have different characteristics regarding color and visibility. The captured near-infrared image is haze-free, but it is grayscale, whereas the visible color image has colors, but it contains haze. There are serious discrepancies in terms of brightness and image structures between the near-infrared image and the visible color image. Due to this discrepancy, the direct use of the near-infrared image for haze removal causes a color distortion problem during near-infrared fusion. The key objective for the near-infrared fusion is therefore to remove the color distortion as well as the haze. To achieve this objective, this paper presents a new near-infrared fusion model that combines the proposed new color and depth regularizations with the conventional haze degradation model. The proposed color regularization sets the color range of the unknown haze-free image based on the combination of the two colors of the colorized near-infrared image and the captured visible color image. That is, the proposed color regularization can provide color information for the unknown haze-free color image. The new depth regularization enables the consecutively estimated depth maps not to be largely deviated, thereby transferring natural-looking colors and high visibility of the colorized near-infrared image into the preliminary dehazed version of the captured visible color image with color distortion and edge artifacts. Experimental results show that the proposed color and depth regularizations can help remove the color distortion and the haze simultaneously. The effectiveness of the proposed color regularization for the near-infrared fusion is verified by comparing it with other conventional regularizations.