MRP-GAN: Multi-resolution parallel generative adversarial networks for text-to-image synthesis

• Published in: Pattern recognition letters Vol. 147; pp. 1 - 7
• Main Authors: Qi, Zhongjian, Fan, Chaogang, Xu, Liangfeng, Li, Xinke, Zhan, Shu
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 01.07.2021; Elsevier Science Ltd
• Subjects: Generative adversarial networks; Image generation; Image quality; Material requirements planning; Semantics; Synthesis; Text-to-image synthesize; Image generation; 41A10; 65D05; Generative adversarial networks; 65D17; Text-to-image synthesize; 41A05
• ISSN: 0167-8655; 1872-7344
• DOI: 10.1016/j.patrec.2021.02.020

•We proposed a new backbone network called multi-resolution parallel structure.•The response gate is proposed to aggregate the outputs of multi-resolution parallel subnetworks.•We apply our proposed above modules to the StackGAN++ to demonstrate the transferability of our method. Synthesizing photographic images from given text descriptions is a challenging problem. Although current methods first synthesize an initial blurred image, then refine the initial image to a high-quality one, the most existing methods are difficult to refine the initial image to an image corresponding to the text description. In this paper, the Multi-resolution Parallel Generative Adversarial Networks for Text-to-Image Synthesis (MRP-GAN) is proposed to generate photographic images. MRP-GAN introduces a Multi-resolution Parallel structure to refine the initial images when the initial images are not synthesized well. The low-resolution semantics are maintained through the whole process by Multi-resolution Parallel structure. Response Gate is designed to fully explore the capability of Multi-resolution Parallel structure by aggregating the outputs of the multi-resolution parallel subnetworks. We also utilize an attention mechanism, named Residual Attention Network, to fine-tune more fine-grained details of the generated images. We evaluate our MRP-GAN model on the CUB and MS-COCO datasets. Extensive experiments demonstrate the state-of-the-art performance of MRP-GAN. Besides, we apply a Multi-resolution Parallel structure in the existing method to verify its transferability.