Toward Controlled Generation of Text

TL;DR

This paper introduces a neural generative model that learns disentangled, interpretable representations for controlled text generation, enabling the production of realistic sentences with specified attributes.

Contribution

It combines variational auto-encoders and attribute discriminators to effectively impose semantic structures in text generation, even with minimal annotations.

Findings

High accuracy in attribute-controlled sentence generation

Effective disentanglement of semantic attributes

Realistic sentence synthesis with desired properties

Abstract

Generic generation and manipulation of text is challenging and has limited success compared to recent deep generative modeling in visual domain. This paper aims at generating plausible natural language sentences, whose attributes are dynamically controlled by learning disentangled latent representations with designated semantics. We propose a new neural generative model which combines variational auto-encoders and holistic attribute discriminators for effective imposition of semantic structures. With differentiable approximation to discrete text samples, explicit constraints on independent attribute controls, and efficient collaborative learning of generator and discriminators, our model learns highly interpretable representations from even only word annotations, and produces realistic sentences with desired attributes. Quantitative evaluation validates the accuracy of sentence and attribute generation.