NEXUS Network: Connecting the Preceding and the Following in Dialogue Generation

TL;DR

This paper introduces the NEXUS Network, a dialogue generation model that enhances response relevance by connecting dialogue history and future conversation through mutual information maximization and a continuous code space.

Contribution

It proposes a novel approach using mutual information and an auxiliary code space to generate more contextually connected and interactive dialogue responses.

Findings

Responses are more relevant to dialogue context

Generated conversations are more interactive

Model outperforms baselines on two datasets

Abstract

Sequence-to-Sequence (seq2seq) models have become overwhelmingly popular in building end-to-end trainable dialogue systems. Though highly efficient in learning the backbone of human-computer communications, they suffer from the problem of strongly favoring short generic responses. In this paper, we argue that a good response should smoothly connect both the preceding dialogue history and the following conversations. We strengthen this connection through mutual information maximization. To sidestep the non-differentiability of discrete natural language tokens, we introduce an auxiliary continuous code space and map such code space to a learnable prior distribution for generation purpose. Experiments on two dialogue datasets validate the effectiveness of our model, where the generated responses are closely related to the dialogue context and lead to more interactive conversations.