WaterGAN: Unsupervised Generative Network to Enable Real-Time Color Correction of Monocular Underwater Images

• Published in: IEEE robotics and automation letters Vol. 3; no. 1; pp. 387 - 394
• Main Authors: Jie Li, Skinner, Katherine A., Eustice, Ryan M., Johnson-Roberson, Matthew
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Attenuation; Autonomous underwater vehicles; Color imagery; Columnar structure; Datasets; Deep sea environments; Gallium nitride; Generators; Image color analysis; Image resolution; Image restoration; Light attenuation; Machine learning; Marine robotics; Mathematical models; Nonlinear systems; Ocean floor; Parameters; Pipelines; Remotely operated vehicles; Restoration; Source code; Training; visual learning; Water circulation; Wave attenuation
• ISSN: 2377-3766; 2377-3766
• DOI: 10.1109/LRA.2017.2730363

This letter reports on WaterGAN, a generative adversarial network (GAN) for generating realistic underwater images from in-air image and depth pairings in an unsupervised pipeline used for color correction of monocular underwater images. Cameras onboard autonomous and remotely operated vehicles can capture high-resolution images to map the seafloor; however, underwater image formation is subject to the complex process of light propagation through the water column. The raw images retrieved are characteristically different than images taken in air due to effects, such as absorption and scattering, which cause attenuation of light at different rates for different wavelengths. While this physical process is well described theoretically, the model depends on many parameters intrinsic to the water column as well as the structure of the scene. These factors make recovery of these parameters difficult without simplifying assumptions or field calibration; hence, restoration of underwater images is a nontrivial problem. Deep learning has demonstrated great success in modeling complex nonlinear systems but requires a large amount of training data, which is difficult to compile in deep sea environments. Using WaterGAN, we generate a large training dataset of corresponding depth, in-air color images, and realistic underwater images. These data serve as input to a two-stage network for color correction of monocular underwater images. Our proposed pipeline is validated with testing on real data collected from both a pure water test tank and from underwater surveys collected in the field. Source code, sample datasets, and pretrained models are made publicly available.