Semi-supervised Bootstrapping of Dialogue State Trackers for Task Oriented Modelling

TL;DR

This paper explores semi-supervised learning techniques to reduce the need for detailed annotations in dialogue state tracking, demonstrating that up to 30% annotation reduction is possible without performance loss.

Contribution

It introduces a semi-supervised approach for dialogue state tracking and presents the first end-to-end dialogue model for the MultiWOZ corpus, reducing annotation requirements.

Findings

Up to 30% reduction in turn-level annotations achievable.

Semi-supervised methods maintain system performance with less labeled data.

First end-to-end dialogue model for MultiWOZ corpus.

Abstract

Dialogue systems benefit greatly from optimizing on detailed annotations, such as transcribed utterances, internal dialogue state representations and dialogue act labels. However, collecting these annotations is expensive and time-consuming, holding back development in the area of dialogue modelling. In this paper, we investigate semi-supervised learning methods that are able to reduce the amount of required intermediate labelling. We find that by leveraging un-annotated data instead, the amount of turn-level annotations of dialogue state can be significantly reduced when building a neural dialogue system. Our analysis on the MultiWOZ corpus, covering a range of domains and topics, finds that annotations can be reduced by up to 30\% while maintaining equivalent system performance. We also describe and evaluate the first end-to-end dialogue model created for the MultiWOZ corpus.