Interactive Text Generation

TL;DR

This paper introduces an interactive text generation task using user simulators and imitation learning, demonstrating that models trained interactively outperform non-interactive models under similar user input budgets.

Contribution

It proposes a novel interactive training framework for text generation models that reduces costs by using user simulators instead of real users.

Findings

Interactive models outperform non-interactive models with the same user input budget.

Training with user simulators improves the quality of generated text.

Imitation Learning effectively trains models in the interactive setting.

Abstract

Users interact with text, image, code, or other editors on a daily basis. However, machine learning models are rarely trained in the settings that reflect the interactivity between users and their editor. This is understandable as training AI models with real users is not only slow and costly, but what these models learn may be specific to user interface design choices. Unfortunately, this means most of the research on text, code, and image generation has focused on non-interactive settings, whereby the model is expected to get everything right without accounting for any input from a user who may be willing to help. We introduce a new Interactive Text Generation task that allows training generation models interactively without the costs of involving real users, by using user simulators that provide edits that guide the model towards a given target text. We train our interactive models using Imitation Learning, and our experiments against competitive non-interactive generation models show that models trained interactively are superior to their non-interactive counterparts, even when all models are given the same budget of user inputs or edits.