A Sequence-to-Sequence Approach to Dialogue State Tracking

TL;DR

This paper introduces Seq2Seq-DU, a novel sequence-to-sequence model for dialogue state tracking that leverages BERT encodings to improve accuracy and handle unseen schemas in task-oriented dialogue systems.

Contribution

It formalizes dialogue state tracking as a sequence-to-sequence problem using BERT-based encoders, enabling joint modeling of intents, slots, and slot values, and handling unseen schemas.

Findings

Outperforms existing methods on multiple benchmark datasets.

Effectively models categorical and non-categorical slots.

Handles unseen schemas with high accuracy.

Abstract

This paper is concerned with dialogue state tracking (DST) in a task-oriented dialogue system. Building a DST module that is highly effective is still a challenging issue, although significant progresses have been made recently. This paper proposes a new approach to dialogue state tracking, referred to as Seq2Seq-DU, which formalizes DST as a sequence-to-sequence problem. Seq2Seq-DU employs two BERT-based encoders to respectively encode the utterances in the dialogue and the descriptions of schemas, an attender to calculate attentions between the utterance embeddings and the schema embeddings, and a decoder to generate pointers to represent the current state of dialogue. Seq2Seq-DU has the following advantages. It can jointly model intents, slots, and slot values; it can leverage the rich representations of utterances and schemas based on BERT; it can effectively deal with categorical and non-categorical slots, and unseen schemas. In addition, Seq2Seq-DU can also be used in the NLU (natural language understanding) module of a dialogue system. Experimental results on benchmark datasets in different settings (SGD, MultiWOZ2.2, MultiWOZ2.1, WOZ2.0, DSTC2, M2M, SNIPS, and ATIS) show that Seq2Seq-DU outperforms the existing methods.