Deep Joint Depth Estimation and Color Correction From Monocular Underwater Images Based on Unsupervised Adaptation Networks

• Published in: IEEE transactions on circuits and systems for video technology Vol. 30; no. 11; pp. 3995 - 4008
• Main Authors: Ye, Xinchen, Li, Zheng, Sun, Baoli, Wang, Zhihui, Xu, Rui, Li, Haojie, Fan, Xin
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.11.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptation; Atmospheric modeling; Color; color correction; depth estimation; domain adapatation; Domains; Estimation; Image color analysis; Machine vision; Robots; Task analysis; Underwater; Underwater image; unsupervised; Visibility; Visualization
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2019.2958950

Degraded visibility and geometrical distortion typically make the underwater vision more intractable than open air vision, which impedes the development of underwater-related machine vision and robotic perception. Therefore, this paper addresses the problem of joint underwater depth estimation and color correction from monocular underwater images, which aims at enjoying the mutual benefits between these two related tasks from a multi-task perspective. Our core ideas lie in our new deep learning architecture. Due to the lack of effective underwater training data, and the weak generalization to the real-world underwater images trained on synthetic data, we consider the problem from a novel perspective of style-level and feature-level adaptation, and propose an unsupervised adaptation network to deal with the joint learning problem. Specifically, a style adaptation network (SAN) is first proposed to learn a style-level transformation to adapt in-air images to the style of underwater domain. Then, we formulate a task network (TN) to jointly estimate the scene depth and correct the color from a single underwater image by learning domain-invariant representations. The whole framework can be trained end-to-end in an adversarial learning manner. Extensive experiments are conducted under air-to-water domain adaptation settings. We show that the proposed method performs favorably against state-of-the-art methods in both depth estimation and color correction tasks.