Gradient-Based Constrained Sampling from Language Models

TL;DR

This paper introduces MuCoLa, a novel non-autoregressive sampling method for language models that incorporates user-defined constraints via an energy function, improving controllability while maintaining fluency.

Contribution

The paper presents MuCoLa, a new constrained sampling technique combining language model likelihood with differentiable constraints using Langevin Dynamics, enabling flexible and effective controlled text generation.

Findings

MuCoLa outperforms baselines in toxicity avoidance and sentiment control.

It effectively integrates hard and soft constraints in text generation.

The method maintains fluency and task performance while satisfying constraints.

Abstract

Large pretrained language models generate fluent text but are notoriously hard to controllably sample from. In this work, we study constrained sampling from such language models: generating text that satisfies user-defined constraints, while maintaining fluency and the model's performance in a downstream task. We propose MuCoLa -- a sampling procedure that combines the log-likelihood of the language model with arbitrary (differentiable) constraints in a single energy function, and then generates samples in a non-autoregressive manner. Specifically, it initializes the entire output sequence with noise and follows a Markov chain defined by Langevin Dynamics using the gradients of the energy function. We evaluate MuCoLa on text generation with soft and hard constraints as well as their combinations obtaining significant improvements over competitive baselines for toxicity avoidance, sentiment control, and keyword-guided generation.