Overview of the Ninth Dialog System Technology Challenge: DSTC9

TL;DR

The paper summarizes the Ninth Dialog System Technology Challenge (DSTC-9), focusing on end-to-end dialog technologies across four tasks, including knowledge access, multi-domain dialog, interactive evaluation, and multi-modal interaction.

Contribution

It provides detailed descriptions of task definitions, datasets, baselines, evaluation setups, and summarizes submitted system results to highlight current state-of-the-art trends.

Findings

Advances in end-to-end dialog modeling for unstructured knowledge access

Improved multi-domain task-oriented dialog systems

Enhanced interactive and multi-modal dialog evaluation methods

Abstract

This paper introduces the Ninth Dialog System Technology Challenge (DSTC-9). This edition of the DSTC focuses on applying end-to-end dialog technologies for four distinct tasks in dialog systems, namely, 1. Task-oriented dialog Modeling with unstructured knowledge access, 2. Multi-domain task-oriented dialog, 3. Interactive evaluation of dialog, and 4. Situated interactive multi-modal dialog. This paper describes the task definition, provided datasets, baselines and evaluation set-up for each track. We also summarize the results of the submitted systems to highlight the overall trends of the state-of-the-art technologies for the tasks.