Causal Micro-Narratives

TL;DR

This paper introduces a new method for classifying causal micro-narratives in text using large language models, demonstrating high accuracy and potential for social science applications.

Contribution

It presents a novel framework for extracting causal micro-narratives with minimal domain-specific resources, validated on US news articles.

Findings

Best LLM achieved 0.87 F1 on detection and 0.71 on classification

Error analysis highlights linguistic ambiguity as a key challenge

Framework applicable to diverse social science research areas

Abstract

We present a novel approach to classify causal micro-narratives from text. These narratives are sentence-level explanations of the cause(s) and/or effect(s) of a target subject. The approach requires only a subject-specific ontology of causes and effects, and we demonstrate it with an application to inflation narratives. Using a human-annotated dataset spanning historical and contemporary US news articles for training, we evaluate several large language models (LLMs) on this multi-label classification task. The best-performing model--a fine-tuned Llama 3.1 8B--achieves F1 scores of 0.87 on narrative detection and 0.71 on narrative classification. Comprehensive error analysis reveals challenges arising from linguistic ambiguity and highlights how model errors often mirror human annotator disagreements. This research establishes a framework for extracting causal micro-narratives from real-world data, with wide-ranging applications to social science research.