Retrieving Climate Change Disinformation by Narrative

TL;DR

This paper introduces SpecFi, a retrieval-based framework for detecting climate change disinformation narratives without relying on fixed taxonomies, effectively handling emerging narratives and demonstrating robustness to narrative complexity.

Contribution

It reformulates narrative detection as a retrieval task, leveraging community summaries and generative models to identify disinformation narratives without predefined labels.

Findings

SpecFi achieves a MAP of 0.505 on CARDS dataset.

Standard retrieval methods degrade significantly on high-variance narratives.

Community summaries align closely with expert taxonomies, surfacing narrative structures.

Abstract

Detecting climate disinformation narratives typically relies on fixed taxonomies, which do not accommodate emerging narratives. Thus, we re-frame narrative detection as a retrieval task: given a narrative's core message as a query, rank texts from a corpus by alignment with that narrative. This formulation requires no predefined label set and can accommodate emerging narratives. We repurpose three climate disinformation datasets (CARDS, Climate Obstruction, climate change subset of PolyNarrative) for retrieval evaluation and propose SpecFi, a framework that generates hypothetical documents to bridge the gap between abstract narrative descriptions and their concrete textual instantiations. SpecFi uses community summaries from graph-based community detection as few-shot examples for generation, achieving a MAP of 0.505 on CARDS without access to narrative labels. We further introduce narrative variance, an embedding-based difficulty metric, and show via partial correlation analysis that standard retrieval degrades on high-variance narratives (BM25 loses 63.4% of MAP), while SpecFi-CS remains robust (32.7% loss). Our analysis also reveals that unsupervised community summaries converge on descriptions close to expert-crafted taxonomies, suggesting that graph-based methods can surface narrative structure from unlabeled text.