NeuralWOZ: Learning to Collect Task-Oriented Dialogue via Model-Based Simulation

TL;DR

NeuralWOZ is a framework that uses model-based simulation with two pipelined models, Collector and Labeler, to generate and annotate task-oriented dialogues, improving zero-shot domain transfer in dialogue state tracking.

Contribution

The paper introduces NeuralWOZ, a novel dialogue collection method utilizing model-based simulation for better zero-shot transfer learning.

Findings

Achieved 4.4% higher joint goal accuracy across domains.

Improved zero-shot coverage by 5.7% on MultiWOZ 2.1.

Generated synthetic dialogues that enhance dialogue state tracking.

Abstract

We propose NeuralWOZ, a novel dialogue collection framework that uses model-based dialogue simulation. NeuralWOZ has two pipelined models, Collector and Labeler. Collector generates dialogues from (1) user's goal instructions, which are the user context and task constraints in natural language, and (2) system's API call results, which is a list of possible query responses for user requests from the given knowledge base. Labeler annotates the generated dialogue by formulating the annotation as a multiple-choice problem, in which the candidate labels are extracted from goal instructions and API call results. We demonstrate the effectiveness of the proposed method in the zero-shot domain transfer learning for dialogue state tracking. In the evaluation, the synthetic dialogue corpus generated from NeuralWOZ achieves a new state-of-the-art with improvements of 4.4% point joint goal accuracy on average across domains, and improvements of 5.7% point of zero-shot coverage against the MultiWOZ 2.1 dataset.