Depth-guided asymmetric CycleGAN for rain synthesis and image deraining

Based on supervised learning, most of the existing single image deraining networks are trained on paired images including one clean image and one rain image. Since it is difficult to obtain a sufficient number of paired images, most of the rain images are manually synthesized from the clean ones. Ho...

Full description

Saved in:
Bibliographic Details
Published inMultimedia tools and applications Vol. 81; no. 25; pp. 35935 - 35952
Main Authors Qi, Yinhe, Zhang, Huanrong, Jin, Zhi, Liu, Wanquan
Format Journal Article
LanguageEnglish
Published New York Springer US 01.10.2022
Springer Nature B.V
Subjects
1190: Depth-Related Processing and Applications in Visual Systems
Algorithms
Asymmetry
Computer Communication Networks
Computer Science
Data Structures and Information Theory
Datasets
Generators
Multimedia
Multimedia Information Systems
Networks
Rain
Special Purpose and Application-Based Systems
Synthesis
Depth information
Asymmetric CycleGAN
Rain synthesis
Deraining
Online AccessGet full text

Cover

More Information
Summary:Based on supervised learning, most of the existing single image deraining networks are trained on paired images including one clean image and one rain image. Since it is difficult to obtain a sufficient number of paired images, most of the rain images are manually synthesized from the clean ones. However, it costs huge time and effort, and requires professional experience to mimic the real rain images well. Moreover, the superior performance of these deraining networks trained on manually synthetic rain images is hard to be maintained when tested on real rain images. In this work, to obtain more realistic rain images for training supervised deraining networks, the depth-guided asymmetric CycleGAN (DA-CycleGAN) is proposed to translate clean images to their rainy counterparts automatically. Due to the cycle consistency strategy, DA-CycleGAN can also implement the single image deraining task unsupervised while synthesizing rain on clean images. Since rain streaks and rain mist vary with depth from the camera, DA-CycleGAN adopts depth information as an aid for rain synthesis and deraining. Furthermore, we design generators with different architectures for these two processes due to the information asymmetry in rain synthesis and deraining. Extensive experiments indicate that the DA-CycleGAN can synthesize more lifelike rain images and provide commensurate deraining performance compared with the state-of-the-art deraining methods.
ISSN:1380-7501
1573-7721
DOI:10.1007/s11042-022-13342-9