ClimateCheck 2026: Scientific Fact-Checking and Disinformation Narrative Classification of Climate-related Claims

TL;DR

ClimateCheck 2026 is a shared task focused on automatically verifying climate claims against scientific literature and classifying disinformation narratives, with expanded data and new evaluation methods.

Contribution

It introduces a new disinformation narrative classification task, expanded dataset, and an automated framework for assessing retrieval quality in climate fact-checking.

Findings

Systems used dense retrieval, cross-encoder ensembles, and large language models.

Evaluation exposed biases in conventional metrics under incomplete annotations.

Not all climate disinformation is equally verifiable, affecting fact-checking system design.

Abstract

Automatically verifying climate-related claims against scientific literature is a challenging task, complicated by the specialised nature of scholarly evidence and the diversity of rhetorical strategies underlying climate disinformation. ClimateCheck 2026 is the second iteration of a shared task addressing this challenge, expanding on the 2025 edition with tripled training data and a new disinformation narrative classification task. Running from January to February 2026 on the CodaBench platform, the competition attracted 20 registered participants and 8 leaderboard submissions, with systems combining dense retrieval pipelines, cross-encoder ensembles, and large language models with structured hierarchical reasoning. In addition to standard evaluation metrics (Recall@K and Binary Preference), we adapt an automated framework to assess retrieval quality under incomplete annotations, exposing systematic biases in how conventional metrics rank systems. A cross-task analysis further reveals that not all climate disinformation is equally verifiable, potentially implicating how future fact-checking systems should be designed.