Toward Unified Controllable Text Generation via Regular Expression Instruction

TL;DR

This paper introduces Regular Expression Instruction (REI), a unified, instruction-based method for controllable text generation that efficiently handles diverse constraints without extensive model modifications.

Contribution

REI provides a novel, unified approach to controllable text generation using regular expression-style instructions, compatible with various constraints and requiring minimal fine-tuning or adjustments.

Findings

High success rates in constraint satisfaction

Effective handling of complex constraint combinations

Outperforms most previous baselines in automatic metrics

Abstract

Controllable text generation is a fundamental aspect of natural language generation, with numerous methods proposed for different constraint types. However, these approaches often require significant architectural or decoding modifications, making them challenging to apply to additional constraints or resolve different constraint combinations. To address this, our paper introduces Regular Expression Instruction (REI), which utilizes an instruction-based mechanism to fully exploit regular expressions' advantages to uniformly model diverse constraints. Specifically, our REI supports all popular fine-grained controllable generation constraints, i.e., lexical, positional, and length, as well as their complex combinations, via regular expression-style instructions. Our method only requires fine-tuning on medium-scale language models or few-shot, in-context learning on large language models, and requires no further adjustment when applied to various constraint combinations. Experiments demonstrate that our straightforward approach yields high success rates and adaptability to various constraints while maintaining competitiveness in automatic metrics and outperforming most previous baselines.