X-RiSAWOZ: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agents

TL;DR

This paper introduces X-RiSAWOZ, a multilingual dialogue dataset created through translation and manual editing, along with a toolkit for efficient dataset creation, enabling development of dialogue agents in multiple languages with limited data.

Contribution

The paper presents a new multilingual dialogue dataset and a toolkit for cost-effective dataset creation, advancing multilingual task-oriented dialogue research.

Findings

High-quality multilingual dataset with over 18,000 utterances per language.

Effective translation and post-editing methodology improves dataset quality.

Baseline dialogue agents demonstrate the utility of the dataset in low-resource settings.

Abstract

Task-oriented dialogue research has mainly focused on a few popular languages like English and Chinese, due to the high dataset creation cost for a new language. To reduce the cost, we apply manual editing to automatically translated data. We create a new multilingual benchmark, X-RiSAWOZ, by translating the Chinese RiSAWOZ to 4 languages: English, French, Hindi, Korean; and a code-mixed English-Hindi language. X-RiSAWOZ has more than 18,000 human-verified dialogue utterances for each language, and unlike most multilingual prior work, is an end-to-end dataset for building fully-functioning agents. The many difficulties we encountered in creating X-RiSAWOZ led us to develop a toolset to accelerate the post-editing of a new language dataset after translation. This toolset improves machine translation with a hybrid entity alignment technique that combines neural with dictionary-based methods, along with many automated and semi-automated validation checks. We establish strong baselines for X-RiSAWOZ by training dialogue agents in the zero- and few-shot settings where limited gold data is available in the target language. Our results suggest that our translation and post-editing methodology and toolset can be used to create new high-quality multilingual dialogue agents cost-effectively. Our dataset, code, and toolkit are released open-source.