ARAML: A Stable Adversarial Training Framework for Text Generation

TL;DR

This paper introduces ARAML, a new adversarial training framework for text generation that stabilizes training by combining maximum likelihood with discriminator rewards, outperforming existing GAN methods.

Contribution

The paper proposes ARAML, a novel framework that replaces policy gradient with reward-augmented maximum likelihood, improving stability and performance in text GAN training.

Findings

Outperforms state-of-the-art text GANs

Provides more stable training process

Achieves better text generation quality

Abstract

Most of the existing generative adversarial networks (GAN) for text generation suffer from the instability of reinforcement learning training algorithms such as policy gradient, leading to unstable performance. To tackle this problem, we propose a novel framework called Adversarial Reward Augmented Maximum Likelihood (ARAML). During adversarial training, the discriminator assigns rewards to samples which are acquired from a stationary distribution near the data rather than the generator's distribution. The generator is optimized with maximum likelihood estimation augmented by the discriminator's rewards instead of policy gradient. Experiments show that our model can outperform state-of-the-art text GANs with a more stable training process.