Controllable Dialogue Simulation with In-Context Learning

TL;DR

This paper introduces extsc{Dialogic}, a cost-effective dialogue simulation method using large language models to generate annotated dialogues, improving low-resource dialogue system training and data augmentation.

Contribution

The paper presents a novel in-context learning based dialogue simulation approach that automates dataset creation with minimal human effort, outperforming traditional crowdsourcing methods.

Findings

Training on simulated dialogues improves task performance in low-resource settings.

The method achieves near-human fluency and annotation accuracy in generated dialogues.

Effective data augmentation with minimal seed data enhances dialogue system training.

Abstract

Building dialogue systems requires a large corpus of annotated dialogues. Such datasets are usually created via crowdsourcing, which is expensive and time-consuming. In this paper, we propose \textsc{Dialogic}, a novel dialogue simulation method based on large language model in-context learning to automate dataset creation. Seeded with a few annotated dialogues, \textsc{Dialogic} automatically selects in-context examples for demonstration and prompts GPT-3 to generate new dialogues and annotations in a controllable way. Our method can rapidly expand a small set of dialogue data with minimum or zero \textit{human involvement} and \textit{parameter update} and is thus much more cost-efficient and time-saving than crowdsourcing. Experimental results on the MultiWOZ dataset demonstrate that training a model on the simulated dialogues leads to even better performance than using the same amount of human-generated dialogues under the challenging low-resource settings, with as few as 85 dialogues as a seed. When enough data is available, our method can still serve as an effective data augmentation method. Human evaluation results also show that our simulated dialogues have near-human fluency and annotation accuracy. The code and data are available at \textbf{\url{https://github.com/Leezekun/dialogic}}.