Diverse Retrieval-Augmented In-Context Learning for Dialogue State Tracking

TL;DR

This paper introduces RefPyDST, a novel approach for dialogue state tracking that formulates the task as Python programming, uses diverse example retrieval, and employs a re-weighting decoding method, achieving state-of-the-art results in zero and few-shot settings.

Contribution

RefPyDST advances in-context learning for DST by modeling it as Python programming, incorporating diverse example retrieval, and applying a re-weighting decoding strategy.

Findings

Achieves state-of-the-art multi-domain joint-goal accuracy on MultiWOZ.

Effective in zero and few-shot learning scenarios.

Improves performance by diverse example retrieval and re-weighting decoding.

Abstract

There has been significant interest in zero and few-shot learning for dialogue state tracking (DST) due to the high cost of collecting and annotating task-oriented dialogues. Recent work has demonstrated that in-context learning requires very little data and zero parameter updates, and even outperforms trained methods in the few-shot setting (Hu et al. 2022). We propose RefPyDST, which advances the state of the art with three advancements to in-context learning for DST. First, we formulate DST as a Python programming task, explicitly modeling language coreference as variable reference in Python. Second, since in-context learning depends highly on the context examples, we propose a method to retrieve a diverse set of relevant examples to improve performance. Finally, we introduce a novel re-weighting method during decoding that takes into account probabilities of competing surface forms, and produces a more accurate dialogue state prediction. We evaluate our approach using MultiWOZ and achieve state-of-the-art multi-domain joint-goal accuracy in zero and few-shot settings.