SAR-to-optical image translation based on improved CGAN

• Published in: Pattern recognition Vol. 121; p. 108208
• Main Authors: Yang, Xi, Zhao, Jingyi, Wei, Ziyu, Wang, Nannan, Gao, Xinbo
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.01.2022
• Subjects: Chromatic aberration loss; ICGAN; SAR-to-optical image translation; SAR-to-optical image translation; ICGAN; Chromatic aberration loss
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2021.108208

•We propose an improved CGAN method for SAR-to-optical image translation.•We propose a parallel feature fusion generator to improve the contour sharpness.•We utilize a multi-scale discriminator to improve the texture fine-grainedness.•We design a chromatic aberration loss to improve the color fidelity of the generated optical image. SAR images have the advantages of being less susceptible to clouds and light, while optical images conform to the human vision system. Both of them are widely applied in the field of scene classification, natural environment monitoring, disaster warning, etc. However, due to the speckle noise caused by the SAR imaging principle, it is difficult for people to distinguish the ground objects from complex background without professional knowledge. One commonly used solution is to exploit Generative Adversarial Networks (GAN) to translate SAR images to optical images which is able to clearly present ground objects with rich color information, i.e., SAR-to-optical image translation. Traditional GAN-based translation methods are apt to cause blurring of contour, disappearance of texture and inconsistency of color. To this end, we propose an improved conditional GAN (ICGAN) method. Compared with the basic CGAN model, the translation ability of our method is improved in the following three aspects. (1) Contour sharpness. We utilize the parallel branches to combine low-level and high-level features, and thus the image contour information is improved without the influence of noise. (2) Texture fine-grainedness. We discriminate the image using multi-scale receptive fields to enrich the local and global texture features of the image. (3) Color fidelity. We use the chromatic aberration loss which is based on Gaussian blur convolution to reduce the color gap between the generated image and the real optical image. Our method considers both the visual layer and the conceptual layer of the image to complete the SAR-to-optical image translation task. The model is able to preserve the contours and textures of the SAR image, while more closely approximates the colors of the ground truth. The experimental results show that the generated image not only has preferable results in visual effects and favorable evaluation metrics (subjective and objective), but also achieves outstanding classification accuracy, which proves the superiority of our method over the state-of-the-arts in the SAR-to-optical image translation task.