Learning End-to-End Goal-Oriented Dialog

TL;DR

This paper evaluates an end-to-end Memory Network-based dialog system for goal-oriented tasks like restaurant reservations, highlighting its capabilities and limitations compared to traditional slot-filling methods.

Contribution

It introduces a testbed for assessing end-to-end dialog systems in goal-oriented settings and demonstrates the system's ability to perform complex operations with promising results.

Findings

Memory Network system achieves promising performance

System can perform non-trivial dialog operations

Compared favorably to slot-filling baseline

Abstract

Traditional dialog systems used in goal-oriented applications require a lot of domain-specific handcrafting, which hinders scaling up to new domains. End-to-end dialog systems, in which all components are trained from the dialogs themselves, escape this limitation. But the encouraging success recently obtained in chit-chat dialog may not carry over to goal-oriented settings. This paper proposes a testbed to break down the strengths and shortcomings of end-to-end dialog systems in goal-oriented applications. Set in the context of restaurant reservation, our tasks require manipulating sentences and symbols, so as to properly conduct conversations, issue API calls and use the outputs of such calls. We show that an end-to-end dialog system based on Memory Networks can reach promising, yet imperfect, performance and learn to perform non-trivial operations. We confirm those results by comparing our system to a hand-crafted slot-filling baseline on data from the second Dialog State Tracking Challenge (Henderson et al., 2014a). We show similar result patterns on data extracted from an online concierge service.