Schema-Guided Dialogue State Tracking Task at DSTC8
Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, Pranav, Khaitan

TL;DR
This paper presents the Schema-Guided Dialogue State Tracking task at DSTC8, focusing on large-scale, data-efficient, and zero-shot generalizable models for virtual assistants, supported by a new dataset and baseline model.
Contribution
It introduces a new large-scale dataset, defines a challenging task for zero-shot API generalization, and provides baseline models and evaluation methods for future research.
Findings
Participants developed neural models exceeding baseline performance
Use of pre-trained encoders and data augmentation improved results
High participation indicates strong interest and progress in the field
Abstract
This paper gives an overview of the Schema-Guided Dialogue State Tracking task of the 8th Dialogue System Technology Challenge. The goal of this task is to develop dialogue state tracking models suitable for large-scale virtual assistants, with a focus on data-efficient joint modeling across domains and zero-shot generalization to new APIs. This task provided a new dataset consisting of over 16000 dialogues in the training set spanning 16 domains to highlight these challenges, and a baseline model capable of zero-shot generalization to new APIs. Twenty-five teams participated, developing a range of neural network models, exceeding the performance of the baseline model by a very high margin. The submissions incorporated a variety of pre-trained encoders and data augmentation techniques. This paper describes the task definition, dataset and evaluation methodology. We also summarize the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Speech and dialogue systems · Natural Language Processing Techniques
