A Benchmark for Understanding and Generating Dialogue between Characters in Stories

TL;DR

This paper introduces DialStory, a new dataset and tasks for machine understanding and generation of story dialogue, emphasizing character traits and relationships, with improved models demonstrating better coherence and accuracy.

Contribution

The paper presents the first dataset and tasks for story dialogue understanding and generation, along with explicit character representations to enhance model performance.

Findings

Models achieve higher coherence and informativeness in dialogue generation.

Explicit character representations improve speaker recognition accuracy.

DialStory dataset enables evaluation of dialogue understanding in stories.

Abstract

Many classical fairy tales, fiction, and screenplays leverage dialogue to advance story plots and establish characters. We present the first study to explore whether machines can understand and generate dialogue in stories, which requires capturing traits of different characters and the relationships between them. To this end, we propose two new tasks including Masked Dialogue Generation and Dialogue Speaker Recognition, i.e., generating missing dialogue turns and predicting speakers for specified dialogue turns, respectively. We build a new dataset DialStory, which consists of 105k Chinese stories with a large amount of dialogue weaved into the plots to support the evaluation. We show the difficulty of the proposed tasks by testing existing models with automatic and manual evaluation on DialStory. Furthermore, we propose to learn explicit character representations to improve performance on these tasks. Extensive experiments and case studies show that our approach can generate more coherent and informative dialogue, and achieve higher speaker recognition accuracy than strong baselines.