PCFG-based Natural Language Interface Improves Generalization for Controlled Text Generation

TL;DR

This paper introduces a PCFG-based natural language interface for controlled text generation, enabling models to better generalize to unseen commands and attribute combinations, surpassing fixed-template methods.

Contribution

It proposes a novel PCFG-based approach to embed control attributes into natural language commands, enhancing generalization in controlled text generation models.

Findings

Effective handling of unseen commands with PCFG-based generation

Models generalize well to unseen attributes and combinations

Simple conditional generation with NL interface is a strong baseline

Abstract

Existing work on controlled text generation (CTG) assumes a control interface of categorical attributes. In this work, we propose a natural language (NL) interface, where we craft a PCFG to embed the control attributes into natural language commands, and propose variants of existing CTG models that take commands as input. In our experiments, we design tailored setups to test model's generalization abilities. We find our PCFG-based command generation approach is effective for handling unseen commands compared to fix-set templates; our proposed NL models can effectively generalize to unseen attributes, a new ability enabled by the NL interface, as well as unseen attribute combinations. Interestingly, we discover that the simple conditional generation approach, enhanced with our proposed NL interface, is a strong baseline in those challenging settings.