Utilizing Amari-Alpha Divergence to Stabilize the Training of Generative Adversarial Networks

• Published in: Entropy (Basel, Switzerland) Vol. 22; no. 4; p. 410
• Main Authors: Cai, Likun, Chen, Yanjie, Cai, Ning, Cheng, Wei, Wang, Hao
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 04.04.2020; MDPI
• Subjects: Alpha divergence; Chi-square test; deep neural networks; generative adversarial network; Generative adversarial networks; Generators; Image processing; Image quality; Image resolution; Machine learning; Optimization; Order parameters; Quality; Stability analysis; Training; unsupervised image generation
• ISSN: 1099-4300; 1099-4300
• DOI: 10.3390/e22040410

Generative Adversarial Nets (GANs) are one of the most popular architectures for image generation, which has achieved significant progress in generating high-resolution, diverse image samples. The normal GANs are supposed to minimize the Kullback–Leibler divergence between distributions of natural and generated images. In this paper, we propose the Alpha-divergence Generative Adversarial Net (Alpha-GAN) which adopts the alpha divergence as the minimization objective function of generators. The alpha divergence can be regarded as a generalization of the Kullback–Leibler divergence, Pearson χ 2 divergence, Hellinger divergence, etc. Our Alpha-GAN employs the power function as the form of adversarial loss for the discriminator with two-order indexes. These hyper-parameters make our model more flexible to trade off between the generated and target distributions. We further give a theoretical analysis of how to select these hyper-parameters to balance the training stability and the quality of generated images. Extensive experiments of Alpha-GAN are performed on SVHN and CelebA datasets, and evaluation results show the stability of Alpha-GAN. The generated samples are also competitive compared with the state-of-the-art approaches.