TVShowGuess: Character Comprehension in Stories as Speaker Guessing

TL;DR

This paper introduces TVShowGuess, a novel task for evaluating machine understanding of fictional characters in TV series scripts, focusing on character identification based on scene backgrounds and dialogues, aligning with human theory of mind.

Contribution

It presents a new character comprehension task, proposes models for encoding long narrative texts, and demonstrates that these models outperform baselines but still fall short of human performance.

Findings

Models outperform baselines significantly.

Models still lag behind human accuracy.

Task effectively assesses understanding of character personas.

Abstract

We propose a new task for assessing machines' skills of understanding fictional characters in narrative stories. The task, TVShowGuess, builds on the scripts of TV series and takes the form of guessing the anonymous main characters based on the backgrounds of the scenes and the dialogues. Our human study supports that this form of task covers comprehension of multiple types of character persona, including understanding characters' personalities, facts and memories of personal experience, which are well aligned with the psychological and literary theories about the theory of mind (ToM) of human beings on understanding fictional characters during reading. We further propose new model architectures to support the contextualized encoding of long scene texts. Experiments show that our proposed approaches significantly outperform baselines, yet still largely lag behind the (nearly perfect) human performance. Our work serves as a first step toward the goal of narrative character comprehension.