Multi-Granularity Representations of Dialog

TL;DR

This paper presents a novel multi-granularity training method for neural dialog models that explicitly learns multiple levels of language representations, improving performance and transferability across dialog tasks.

Contribution

It introduces a new training procedure that controls the granularity of learned representations by modifying negative sampling, enhancing dialog model capabilities.

Findings

Improved next utterance retrieval performance on MultiWOZ and Ubuntu datasets.

Demonstrated learning of multiple granularities of representations.

Enhanced transferability to downstream dialog tasks.

Abstract

Neural models of dialog rely on generalized latent representations of language. This paper introduces a novel training procedure which explicitly learns multiple representations of language at several levels of granularity. The multi-granularity training algorithm modifies the mechanism by which negative candidate responses are sampled in order to control the granularity of learned latent representations. Strong performance gains are observed on the next utterance retrieval task using both the MultiWOZ dataset and the Ubuntu dialog corpus. Analysis significantly demonstrates that multiple granularities of representation are being learned, and that multi-granularity training facilitates better transfer to downstream tasks.