Continuously Learning Neural Dialogue Management

TL;DR

This paper presents a neural dialogue management system that learns from data and continuously improves through reinforcement learning, enhancing task-oriented spoken dialogue performance in noisy environments.

Contribution

It introduces a unified neural network framework for supervised learning and reinforcement learning in dialogue management, enabling continuous improvement within a single model.

Findings

Supervised model performs well in corpus-based evaluation.

Reinforcement learning enhances performance in interactive settings.

Model is robust under high-noise conditions.

Abstract

We describe a two-step approach for dialogue management in task-oriented spoken dialogue systems. A unified neural network framework is proposed to enable the system to first learn by supervision from a set of dialogue data and then continuously improve its behaviour via reinforcement learning, all using gradient-based algorithms on one single model. The experiments demonstrate the supervised model's effectiveness in the corpus-based evaluation, with user simulation, and with paid human subjects. The use of reinforcement learning further improves the model's performance in both interactive settings, especially under higher-noise conditions.