Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models

TL;DR

This paper introduces ARDM, a dialog model leveraging large pre-trained language models that models speakers separately, requiring no human annotations, and performs well on task-oriented and persuasion dialog tasks.

Contribution

The paper presents ARDM, a novel framework that effectively utilizes large pre-trained language models for dialog generation without supervision from human annotations.

Findings

ARDM outperforms or matches state-of-the-art on CamRest676 and MultiWOZ datasets.

ARDM generalizes to persuasion tasks, generating human-like persuasive responses.

No human annotation supervision needed for effective dialog modeling.

Abstract

Existing dialog system models require extensive human annotations and are difficult to generalize to different tasks. The recent success of large pre-trained language models such as BERT and GPT-2 (Devlin et al., 2019; Radford et al., 2019) have suggested the effectiveness of incorporating language priors in down-stream NLP tasks. However, how much pre-trained language models can help dialog response generation is still under exploration. In this paper, we propose a simple, general, and effective framework: Alternating Roles Dialog Model (ARDM). ARDM models each speaker separately and takes advantage of the large pre-trained language model. It requires no supervision from human annotations such as belief states or dialog acts to achieve effective conversations. ARDM outperforms or is on par with state-of-the-art methods on two popular task-oriented dialog datasets: CamRest676 and MultiWOZ. Moreover, we can generalize ARDM to more challenging, non-collaborative tasks such as persuasion. In persuasion tasks, ARDM is capable of generating human-like responses to persuade people to donate to a charity.