Tackling mode collapse in multi-generator GANs with orthogonal vectors

• Published in: Pattern recognition Vol. 110; p. 107646
• Main Authors: Li, Wei, Fan, Li, Wang, Zhenyu, Ma, Chao, Cui, Xiaohui
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.02.2021
• Subjects: GANs; Minimax formula; Mode collapse; Multiple generators; Orthogonal vectors; Multiple generators; Mode collapse; Orthogonal vectors; GANs; Minimax formula
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2020.107646

•This paper proposes a novel MGO-GAN model which learns a mapping function parameterized by multiple generators from the randomized space to the original data space, overcoming the problem of mode collapse.•This paper utilizes the back propagation to minimize the orthogonal value in GAN and combine the orthogonal value with the generator loss to jointly update the parameters of generator from both theoretical and empirical perspectives, offering new insights into the success of MGO-GAN.•Through comprehensive experiments on three datasets with different resolutions, we demonstrate the effectiveness of the proposed approach. Generative Adversarial Networks (GANs) have been widely used to generate realistic-looking instances. However, training robust GAN is a non-trivial task due to the problem of mode collapse. Although many GAN variants are proposed to overcome this problem, they have limitations. Those existing studies either generate identical instances or result in negative gradients during training. In this paper, we propose a new approach to training GAN to overcome mode collapse by employing a set of generators, an encoder and a discriminator. A new minimax formula is proposed to simultaneously train all components in a similar spirit to vanilla GAN. The orthogonal vector strategy is employed to guide multiple generators to learn different information in a complementary manner. In this way, we term our approach Multi-Generator Orthogonal GAN (MGO-GAN). Specifically, the synthetic data produced by those generators are fed into the encoder to obtain feature vectors. The orthogonal value is calculated between any two feature vectors, which loyally reflects the correlation between vectors. Such a correlation indicates how different information has been learnt by generators. The lower the orthogonal value is, the more different information the generators learn. We minimize the orthogonal value along with minimizing the generator loss through back-propagation in the training of GAN. The orthogonal value is integrated with the original generator loss to jointly update the corresponding generator’s parameters. We conduct extensive experiments utilizing MNIST, CIFAR10 and CelebA datasets to demonstrate the significant performance improvement of MGO-GAN in terms of generated data quality and diversity at different resolutions.