Deep Reinforcement Learning for On-line Dialogue State Tracking

TL;DR

This paper introduces a novel deep reinforcement learning framework for on-line dialogue state tracking that improves dialogue management performance and allows joint optimization of dialogue policy.

Contribution

It is the first to apply DRL for on-line DST optimization and enables joint training of DST and dialogue policy.

Findings

On-line DST optimization improves dialogue manager performance.

Joint training of DST and policy yields further improvements.

Framework maintains flexibility of predefined policies.

Abstract

Dialogue state tracking (DST) is a crucial module in dialogue management. It is usually cast as a supervised training problem, which is not convenient for on-line optimization. In this paper, a novel companion teaching based deep reinforcement learning (DRL) framework for on-line DST optimization is proposed. To the best of our knowledge, this is the first effort to optimize the DST module within DRL framework for on-line task-oriented spoken dialogue systems. In addition, dialogue policy can be further jointly updated. Experiments show that on-line DST optimization can effectively improve the dialogue manager performance while keeping the flexibility of using predefined policy. Joint training of both DST and policy can further improve the performance.