Adversarial Text Generation Without Reinforcement Learning

TL;DR

This paper introduces a novel approach for adversarial text generation that leverages an autoencoder to enable GAN training without reinforcement learning, resulting in realistic and high-quality language output.

Contribution

The paper presents a method combining autoencoders with GANs to generate natural language without reinforcement learning, improving training efficiency and output quality.

Findings

Generated text is realistic and competitive with baselines.

Latent space visualization confirms proper learning of sentence representations.

Model produces coherent and diverse sentences, validated by human ratings and BLEU scores.

Abstract

Generative Adversarial Networks (GANs) have experienced a recent surge in popularity, performing competitively in a variety of tasks, especially in computer vision. However, GAN training has shown limited success in natural language processing. This is largely because sequences of text are discrete, and thus gradients cannot propagate from the discriminator to the generator. Recent solutions use reinforcement learning to propagate approximate gradients to the generator, but this is inefficient to train. We propose to utilize an autoencoder to learn a low-dimensional representation of sentences. A GAN is then trained to generate its own vectors in this space, which decode to realistic utterances. We report both random and interpolated samples from the generator. Visualization of sentence vectors indicate our model correctly learns the latent space of the autoencoder. Both human ratings and BLEU scores show that our model generates realistic text against competitive baselines.