Multi-task learning for Joint Language Understanding and Dialogue State Tracking

TL;DR

This paper introduces a multi-task learning framework that jointly improves language understanding and dialogue state tracking in task-oriented dialogue systems by sharing neural network components and handling large or unseen slot values.

Contribution

It proposes a novel multi-task training approach that shares encoding layers and uses candidate sets for DST, effectively managing large or unseen slot values and reducing model complexity.

Findings

Enhanced performance in LU and DST tasks.

Effective handling of large and unseen slot values.

Reduced network parameters through shared training.

Abstract

This paper presents a novel approach for multi-task learning of language understanding (LU) and dialogue state tracking (DST) in task-oriented dialogue systems. Multi-task training enables the sharing of the neural network layers responsible for encoding the user utterance for both LU and DST and improves performance while reducing the number of network parameters. In our proposed framework, DST operates on a set of candidate values for each slot that has been mentioned so far. These candidate sets are generated using LU slot annotations for the current user utterance, dialogue acts corresponding to the preceding system utterance and the dialogue state estimated for the previous turn, enabling DST to handle slots with a large or unbounded set of possible values and deal with slot values not seen during training. Furthermore, to bridge the gap between training and inference, we investigate the use of scheduled sampling on LU output for the current user utterance as well as the DST output for the preceding turn.