EvolvTrip: Enhancing Literary Character Understanding with Temporal Theory-of-Mind Graphs

TL;DR

This paper introduces EvolvTrip, a temporal knowledge graph that improves large language models' ability to understand characters' evolving mental states in long narratives, addressing a key challenge in narrative comprehension.

Contribution

EvolvTrip is a novel perspective-aware temporal knowledge graph that enhances LLMs' theory-of-mind reasoning in complex, long-form narratives, especially benefiting smaller models.

Findings

EvolvTrip improves LLM performance on character understanding tasks.

EvolvTrip is particularly effective for smaller models.

EvolvTrip enhances comprehension in extended narratives.

Abstract

A compelling portrayal of characters is essential to the success of narrative writing. For readers, appreciating a character's traits requires the ability to infer their evolving beliefs, desires, and intentions over the course of a complex storyline, a cognitive skill known as Theory-of-Mind (ToM). Performing ToM reasoning in prolonged narratives requires readers to integrate historical context with current narrative information, a task at which humans excel but Large Language Models (LLMs) often struggle. To systematically evaluate LLMs' ToM reasoning capability in long narratives, we construct LitCharToM, a benchmark of character-centric questions across four ToM dimensions from classic literature. Further, we introduce EvolvTrip, a perspective-aware temporal knowledge graph that tracks psychological development throughout narratives. Our experiments demonstrate that EvolvTrip consistently enhances performance of LLMs across varying scales, even in challenging extended-context scenarios. EvolvTrip proves to be particularly valuable for smaller models, partially bridging the performance gap with larger LLMs and showing great compatibility with lengthy narratives. Our findings highlight the importance of explicit representation of temporal character mental states in narrative comprehension and offer a foundation for more sophisticated character understanding. Our data and code are publicly available at https://github.com/Bernard-Yang/EvolvTrip.