AttentionGAN: Unpaired Image-to-Image Translation Using Attention-Guided Generative Adversarial Networks

• Published in: IEEE transaction on neural networks and learning systems Vol. 34; no. 4; pp. 1972 - 1987
• Main Authors: Tang, Hao, Liu, Hong, Xu, Dan, Torr, Philip H. S., Sebe, Nicu
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.04.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Attention guided; Computational modeling; Domains; Generative adversarial networks; generative adversarial networks (GANs); Generators; Image quality; Masks; Semantics; Target masking; Task analysis; Training; Training data; Translation; unpaired image-to-image translation
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2021.3105725

State-of-the-art methods in the image-to-image translation are capable of learning a mapping from a source domain to a target domain with unpaired image data. Though the existing methods have achieved promising results, they still produce visual artifacts, being able to translate low-level information but not high-level semantics of input images. One possible reason is that generators do not have the ability to perceive the most discriminative parts between the source and target domains, thus making the generated images low quality. In this article, we propose a new Attention-Guided Generative Adversarial Networks (AttentionGAN) for the unpaired image-to-image translation task. AttentionGAN can identify the most discriminative foreground objects and minimize the change of the background. The attention-guided generators in AttentionGAN are able to produce attention masks, and then fuse the generation output with the attention masks to obtain high-quality target images. Accordingly, we also design a novel attention-guided discriminator which only considers attended regions. Extensive experiments are conducted on several generative tasks with eight public datasets, demonstrating that the proposed method is effective to generate sharper and more realistic images compared with existing competitive models. The code is available at https://github.com/Ha0Tang/AttentionGAN .