AutoTemplate: A Simple Recipe for Lexically Constrained Text Generation

TL;DR

AutoTemplate is a novel framework for lexically constrained text generation that separates template creation and lexicalization, achieving better quality and constraint satisfaction than existing methods.

Contribution

It introduces a simple two-step approach dividing template generation and lexicalization, improving performance on constrained text generation tasks.

Findings

Outperforms baseline methods on keywords-to-sentence and summarization tasks.

Satisfies all lexical constraints while maintaining high text quality.

Effective in both constrained generation and entity-guided summarization.

Abstract

Lexically constrained text generation is one of the constrained text generation tasks, which aims to generate text that covers all the given constraint lexicons. While the existing approaches tackle this problem using a lexically constrained beam search algorithm or dedicated model using non-autoregressive decoding, there is a trade-off between the generated text quality and the hard constraint satisfaction. We introduce AutoTemplate, a simple yet effective lexically constrained text generation framework divided into template generation and lexicalization tasks. The template generation is to generate the text with the placeholders, and lexicalization replaces them into the constraint lexicons to perform lexically constrained text generation. We conducted the experiments on two tasks: keywords-to-sentence generations and entity-guided summarization. Experimental results show that the AutoTemplate outperforms the competitive baselines on both tasks while satisfying the hard lexical constraints. The code is available at https://github.com/megagonlabs/autotemplate