Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning

TL;DR

This paper introduces an end-to-end deep reinforcement learning framework for dialog state tracking and management that jointly learns language understanding and dialog strategies, demonstrating improved performance in a conversational game simulator.

Contribution

The paper proposes a novel hybrid deep reinforcement learning approach that combines reinforcement and supervised learning for faster dialog policy learning.

Findings

Outperforms modular baseline in a conversational game

Learns a distributed representation of dialog state

Achieves faster learning speed with hybrid algorithm

Abstract

This paper presents an end-to-end framework for task-oriented dialog systems using a variant of Deep Recurrent Q-Networks (DRQN). The model is able to interface with a relational database and jointly learn policies for both language understanding and dialog strategy. Moreover, we propose a hybrid algorithm that combines the strength of reinforcement learning and supervised learning to achieve faster learning speed. We evaluated the proposed model on a 20 Question Game conversational game simulator. Results show that the proposed method outperforms the modular-based baseline and learns a distributed representation of the latent dialog state.