N-Shot Learning for Augmenting Task-Oriented Dialogue State Tracking

TL;DR

This paper presents a belief state annotation-based augmentation framework for task-oriented dialogue state tracking, improving model adaptation with minimal data and enhancing robustness to unseen values.

Contribution

It introduces a novel augmentation method leveraging belief state annotations, effective with as few as five examples, for better dialogue state tracking in low-resource settings.

Findings

Significant performance improvements on TRADE and TOD-BERT models.

Enhanced robustness to unseen dialogue values.

Better performance on seen values during training.

Abstract

Augmentation of task-oriented dialogues has followed standard methods used for plain-text such as back-translation, word-level manipulation, and paraphrasing despite its richly annotated structure. In this work, we introduce an augmentation framework that utilizes belief state annotations to match turns from various dialogues and form new synthetic dialogues in a bottom-up manner. Unlike other augmentation strategies, it operates with as few as five examples. Our augmentation strategy yields significant improvements when both adapting a DST model to a new domain, and when adapting a language model to the DST task, on evaluations with TRADE and TOD-BERT models. Further analysis shows that our model performs better on seen values during training, and it is also more robust to unseen values. We conclude that exploiting belief state annotations enhances dialogue augmentation and results in improved models in n-shot training scenarios.