Robust Dialog State Tracking for Large Ontologies

TL;DR

This paper presents a robust dialog state tracking method for large ontologies that performs well without spoken language understanding output, using string matching and coreference resolution, achieving top results in DSTC 4.

Contribution

The paper introduces a novel dialog state tracking approach tailored for large ontologies and subdialog labeling, outperforming previous methods in DSTC 4.

Findings

Achieved highest F1-score in DSTC 4 among 7 teams

Outperformed the runner-up by 9 and 7 percentage points

Effectively handles large ontologies and subdialog labels

Abstract

The Dialog State Tracking Challenge 4 (DSTC 4) differentiates itself from the previous three editions as follows: the number of slot-value pairs present in the ontology is much larger, no spoken language understanding output is given, and utterances are labeled at the subdialog level. This paper describes a novel dialog state tracking method designed to work robustly under these conditions, using elaborate string matching, coreference resolution tailored for dialogs and a few other improvements. The method can correctly identify many values that are not explicitly present in the utterance. On the final evaluation, our method came in first among 7 competing teams and 24 entries. The F1-score achieved by our method was 9 and 7 percentage points higher than that of the runner-up for the utterance-level evaluation and for the subdialog-level evaluation, respectively.