Adversarial Generation of Natural Language

TL;DR

This paper explores the use of GANs for natural language generation, introducing a simple baseline that achieves state-of-the-art results in Chinese poem generation without relying on gradient estimators.

Contribution

It presents a novel GAN-based approach for natural language generation that overcomes the discrete output challenge without gradient estimators, advancing the field beyond previous likelihood methods.

Findings

Achieved state-of-the-art results on Chinese poem generation dataset

Demonstrated effective sentence generation from grammars

Developed a conditional model for sequence generation based on sentence features

Abstract

Generative Adversarial Networks (GANs) have gathered a lot of attention from the computer vision community, yielding impressive results for image generation. Advances in the adversarial generation of natural language from noise however are not commensurate with the progress made in generating images, and still lag far behind likelihood based methods. In this paper, we take a step towards generating natural language with a GAN objective alone. We introduce a simple baseline that addresses the discrete output space problem without relying on gradient estimators and show that it is able to achieve state-of-the-art results on a Chinese poem generation dataset. We present quantitative results on generating sentences from context-free and probabilistic context-free grammars, and qualitative language modeling results. A conditional version is also described that can generate sequences conditioned on sentence characteristics.