Two Approaches to Building Collaborative, Task-Oriented Dialog Agents through Self-Play

TL;DR

This paper explores two self-play methods, reinforcement learning and game theory, for training task-oriented dialog agents, enabling autonomous discovery of communication strategies in API environments.

Contribution

It introduces and empirically evaluates two novel self-play approaches for training dialog agents without relying on large human dialog datasets.

Findings

Reinforcement learning approach effectively trains dialog agents through self-play.

Game-theoretic equilibrium finding provides a stable strategy for dialog agents.

Both methods enable agents to autonomously develop communication strategies.

Abstract

Task-oriented dialog systems are often trained on human/human dialogs, such as collected from Wizard-of-Oz interfaces. However, human/human corpora are frequently too small for supervised training to be effective. This paper investigates two approaches to training agent-bots and user-bots through self-play, in which they autonomously explore an API environment, discovering communication strategies that enable them to solve the task. We give empirical results for both reinforcement learning and game-theoretic equilibrium finding.