GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution

Generative Adversarial Networks (GAN) have limitations when the goal is to generate sequences of discrete elements. The reason for this is that samples from a distribution on discrete objects such as the multinomial are not differentiable with respect to the distribution parameters. This problem can...

Full description

Saved in:
Bibliographic Details
Main Authors Kusner, Matt J, Hernández-Lobato, José Miguel
Format Journal Article
LanguageEnglish
Published 12.11.2016
Subjects
Computer Science - Learning
Statistics - Machine Learning
Online AccessGet full text

Cover

More Information
Summary:Generative Adversarial Networks (GAN) have limitations when the goal is to generate sequences of discrete elements. The reason for this is that samples from a distribution on discrete objects such as the multinomial are not differentiable with respect to the distribution parameters. This problem can be avoided by using the Gumbel-softmax distribution, which is a continuous approximation to a multinomial distribution parameterized in terms of the softmax function. In this work, we evaluate the performance of GANs based on recurrent neural networks with Gumbel-softmax output distributions in the task of generating sequences of discrete elements.
DOI:10.48550/arxiv.1611.04051