In-Context Learning for Few-Shot Dialogue State Tracking

TL;DR

This paper introduces an in-context learning framework for dialogue state tracking that leverages large language models with exemplars, reformulating the task as text-to-SQL, and demonstrates superior performance in zero and few-shot scenarios.

Contribution

The work presents a novel ICL approach for DST that reformulates the task as text-to-SQL and employs a retrieval method for exemplars, achieving state-of-the-art results in low-resource settings.

Findings

Outperforms previous models in few-shot settings on MultiWOZ

Significantly better in zero-shot scenarios with only task instructions

Reformulating DST as text-to-SQL improves model understanding

Abstract

Collecting and annotating task-oriented dialogues is time-consuming and costly; thus, zero and few shot learning could greatly benefit dialogue state tracking (DST). In this work, we propose an in-context learning (ICL) framework for zero-shot and few-shot learning DST, where a large pre-trained language model (LM) takes a test instance and a few exemplars as input, and directly decodes the dialogue state without any parameter updates. To better leverage a tabular domain description in the LM prompt, we reformulate DST into a text-to-SQL problem. We also propose a novel approach to retrieve annotated dialogues as exemplars. Empirical results on MultiWOZ show that our method IC-DST substantially outperforms previous fine-tuned state-of-the-art models in few-shot settings. In addition, we test IC-DST in zero-shot settings, in which the model only takes a fixed task instruction as input, finding that it outperforms previous zero-shot methods by a large margin.