Enhancing Structural Mapping with LLM-derived Abstractions for Analogical Reasoning in Narratives

TL;DR

This paper introduces YARN, a modular framework that enhances narrative analogical reasoning by using LLM-derived abstractions to improve structural mapping and understanding of stories.

Contribution

The paper presents a novel framework that leverages LLMs to decompose and abstract narratives, significantly improving analogical reasoning performance over baseline models.

Findings

Abstractions improve model performance in narrative reasoning.

YARN outperforms end-to-end LLM baselines in experiments.

Error analysis highlights challenges in abstraction levels and implicit causality.

Abstract

Analogical reasoning is a key driver of human generalization in problem-solving and argumentation. Yet, analogies between narrative structures remain challenging for machines. Cognitive engines for structural mapping are not directly applicable, as they assume pre-extracted entities, whereas LLMs' performance is sensitive to prompt format and the degree of surface similarity between narratives. This gap motivates a key question: What is the impact of enhancing structural mapping with LLM-derived abstractions on their analogical reasoning ability in narratives? To that end, we propose a modular framework named YARN (Yielding Abstractions for Reasoning in Narratives), which uses LLMs to decompose narratives into units, abstract these units, and then passes them to a mapping component that aligns elements across stories to perform analogical reasoning. We define and operationalize four levels of abstraction that capture both the general meaning of units and their roles in the story, grounded in prior work on framing. Our experiments reveal that abstractions consistently improve model performance, resulting in competitive or better performance than end-to-end LLM baselines. Closer error analysis reveals the remaining challenges in abstraction at the right level, in incorporating implicit causality, and an emerging categorization of analogical patterns in narratives. YARN enables systematic variation of experimental settings to analyze component contributions, and to support future work, we make the code for YARN openly available.