Converse, Focus and Guess -- Towards Multi-Document Driven Dialogue

TL;DR

This paper introduces the MD3 task and dataset for multi-document driven dialogue, proposing a model that effectively guesses target documents through dialogue, outperforming baselines and nearing human performance.

Contribution

It presents a new task, dataset, and model for multi-document dialogue that can guess user target documents efficiently by integrating external knowledge and dialogue state tracking.

Findings

The MD3 model outperforms baseline methods.

The model approaches human-level performance.

Effective dialogue policy optimization reduces guessing turns.

Abstract

We propose a novel task, Multi-Document Driven Dialogue (MD3), in which an agent can guess the target document that the user is interested in by leading a dialogue. To benchmark progress, we introduce a new dataset of GuessMovie, which contains 16,881 documents, each describing a movie, and associated 13,434 dialogues. Further, we propose the MD3 model. Keeping guessing the target document in mind, it converses with the user conditioned on both document engagement and user feedback. In order to incorporate large-scale external documents into the dialogue, it pretrains a document representation which is sensitive to attributes it talks about an object. Then it tracks dialogue state by detecting evolvement of document belief and attribute belief, and finally optimizes dialogue policy in principle of entropy decreasing and reward increasing, which is expected to successfully guess the user's target in a minimum number of turns. Experiments show that our method significantly outperforms several strong baseline methods and is very close to human's performance.