Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models

TL;DR

This paper explores building open-domain dialogue systems with generative hierarchical neural networks, demonstrating their competitiveness and ways to improve performance through data and embeddings.

Contribution

It extends hierarchical neural encoder-decoder models to dialogue systems and shows how to enhance their performance with larger data and pretrained embeddings.

Findings

Hierarchical neural models are competitive with state-of-the-art methods.

Bootstrapping with larger datasets improves response quality.

Pretrained embeddings enhance model performance.

Abstract

We investigate the task of building open domain, conversational dialogue systems based on large dialogue corpora using generative models. Generative models produce system responses that are autonomously generated word-by-word, opening up the possibility for realistic, flexible interactions. In support of this goal, we extend the recently proposed hierarchical recurrent encoder-decoder neural network to the dialogue domain, and demonstrate that this model is competitive with state-of-the-art neural language models and back-off n-gram models. We investigate the limitations of this and similar approaches, and show how its performance can be improved by bootstrapping the learning from a larger question-answer pair corpus and from pretrained word embeddings.