Sequence to Backward and Forward Sequences: A Content-Introducing Approach to Generative Short-Text Conversation

TL;DR

This paper introduces a novel content-aware neural dialogue generation model that predicts keywords and generates replies containing these keywords, significantly improving response relevance and diversity over traditional models.

Contribution

The paper proposes a sequence-to-backward and forward sequences model with keyword prediction to enhance the relevance and informativeness of generated dialogue responses.

Findings

Outperforms traditional seq2seq models in human evaluation

Produces more meaningful and diverse replies

Keyword placement is effectively controlled in responses

Abstract

Using neural networks to generate replies in human-computer dialogue systems is attracting increasing attention over the past few years. However, the performance is not satisfactory: the neural network tends to generate safe, universally relevant replies which carry little meaning. In this paper, we propose a content-introducing approach to neural network-based generative dialogue systems. We first use pointwise mutual information (PMI) to predict a noun as a keyword, reflecting the main gist of the reply. We then propose seq2BF, a "sequence to backward and forward sequences" model, which generates a reply containing the given keyword. Experimental results show that our approach significantly outperforms traditional sequence-to-sequence models in terms of human evaluation and the entropy measure, and that the predicted keyword can appear at an appropriate position in the reply.