PCAE: A Framework of Plug-in Conditional Auto-Encoder for Controllable Text Generation

TL;DR

PCAE introduces a flexible, semi-supervised framework for controllable text generation that efficiently incorporates multiple conditions using a plug-in auto-encoder approach, demonstrated across various tasks and models.

Contribution

The paper presents a novel plug-in auto-encoder framework with a broadcasting label fusion network for enhanced controllability and efficiency in text generation.

Findings

Effective across 2 to 10 conditions in multiple tasks

Works with both RNN-based and pre-trained BART auto-encoders

Achieves highly manipulable, diverse, and time-efficient generation

Abstract

Controllable text generation has taken a gigantic step forward these days. Yet existing methods are either constrained in a one-off pattern or not efficient enough for receiving multiple conditions at every generation stage. We propose a model-agnostic framework Plug-in Conditional Auto-Encoder for Controllable Text Generation (PCAE) towards flexible and semi-supervised text generation. Our framework is "plug-and-play" with partial parameters to be fine-tuned in the pre-trained model (less than a half). Crucial to the success of PCAE is the proposed broadcasting label fusion network for navigating the global latent code to a specified local and confined space. Visualization of the local latent prior well confirms the primary devotion in hidden space of the proposed model. Moreover, extensive experiments across five related generation tasks (from 2 conditions up to 10 conditions) on both RNN- based and pre-trained BART [26] based auto-encoders reveal the high capability of PCAE, which enables generation that is highly manipulable, syntactically diverse and time-saving with minimum labeled samples. We will release our code at https://github.com/ImKeTT/pcae.