Diffusion Guided Language Modeling

TL;DR

This paper introduces a guided diffusion approach that combines the fluency of auto-regressive models with the flexibility of diffusion for attribute-controlled text generation, outperforming previous methods.

Contribution

It presents a novel framework that uses diffusion to steer auto-regressive language models, enabling flexible attribute control with minimal additional training.

Findings

Outperforms previous plug-and-play guidance methods on benchmark datasets.

Controls new attributes with a single logistic regression classifier.

Maintains high fluency while guiding attribute-specific text generation.

Abstract

Current language models demonstrate remarkable proficiency in text generation. However, for many applications it is desirable to control attributes, such as sentiment, or toxicity, of the generated language -- ideally tailored towards each specific use case and target audience. For auto-regressive language models, existing guidance methods are prone to decoding errors that cascade during generation and degrade performance. In contrast, text diffusion models can easily be guided with, for example, a simple linear sentiment classifier -- however they do suffer from significantly higher perplexity than auto-regressive alternatives. In this paper we use a guided diffusion model to produce a latent proposal that steers an auto-regressive language model to generate text with desired properties. Our model inherits the unmatched fluency of the auto-regressive approach and the plug-and-play flexibility of diffusion. We show that it outperforms previous plug-and-play guidance methods across a wide range of benchmark data sets. Further, controlling a new attribute in our framework is reduced to training a single logistic regression classifier.