Show, Don't Tell: Uncovering Implicit Character Portrayal using LLMs

TL;DR

This paper introduces LIIPA, a novel framework leveraging large language models to uncover implicit character portrayals in fiction, outperforming existing methods and revealing fairness-accuracy tradeoffs in narrative analysis.

Contribution

The paper presents LIIPA, a new LLM-based approach for analyzing implicit character portrayals, along with a novel dataset and insights into fairness considerations.

Findings

LIIPA outperforms existing approaches in character portrayal analysis.

LIIPA is more robust to increasing character counts.

All LIIPA variants outperform non-LLM baselines in fairness and accuracy.

Abstract

Tools for analyzing character portrayal in fiction are valuable for writers and literary scholars in developing and interpreting compelling stories. Existing tools, such as visualization tools for analyzing fictional characters, primarily rely on explicit textual indicators of character attributes. However, portrayal is often implicit, revealed through actions and behaviors rather than explicit statements. We address this gap by leveraging large language models (LLMs) to uncover implicit character portrayals. We start by generating a dataset for this task with greater cross-topic similarity, lexical diversity, and narrative lengths than existing narrative text corpora such as TinyStories and WritingPrompts. We then introduce LIIPA (LLMs for Inferring Implicit Portrayal for Character Analysis), a framework for prompting LLMs to uncover character portrayals. LIIPA can be configured to use various types of intermediate computation (character attribute word lists, chain-of-thought) to infer how fictional characters are portrayed in the source text. We find that LIIPA outperforms existing approaches, and is more robust to increasing character counts (number of unique persons depicted) due to its ability to utilize full narrative context. Lastly, we investigate the sensitivity of portrayal estimates to character demographics, identifying a fairness-accuracy tradeoff among methods in our LIIPA framework -- a phenomenon familiar within the algorithmic fairness literature. Despite this tradeoff, all LIIPA variants consistently outperform non-LLM baselines in both fairness and accuracy. Our work demonstrates the potential benefits of using LLMs to analyze complex characters and to better understand how implicit portrayal biases may manifest in narrative texts.