Automated Fact-Checking of Climate Change Claims with Large Language Models

TL;DR

Climinator is an AI tool that uses large language models within a Mediator-Advocate framework to automate fact-checking of climate change claims, synthesizing scientific perspectives for accurate, evidence-based conclusions.

Contribution

This work introduces Climinator, a novel AI system employing LLMs and a Mediator-Advocate framework for effective climate claim fact-checking and perspective synthesis.

Findings

Climinator achieves high accuracy on climate claim datasets.

The framework reliably converges to scientific consensus.

It effectively reconciles diverse scientific viewpoints.

Abstract

This paper presents Climinator, a novel AI-based tool designed to automate the fact-checking of climate change claims. Utilizing an array of Large Language Models (LLMs) informed by authoritative sources like the IPCC reports and peer-reviewed scientific literature, Climinator employs an innovative Mediator-Advocate framework. This design allows Climinator to effectively synthesize varying scientific perspectives, leading to robust, evidence-based evaluations. Our model demonstrates remarkable accuracy when testing claims collected from Climate Feedback and Skeptical Science. Notably, when integrating an advocate with a climate science denial perspective in our framework, Climinator's iterative debate process reliably converges towards scientific consensus, underscoring its adeptness at reconciling diverse viewpoints into science-based, factual conclusions. While our research is subject to certain limitations and necessitates careful interpretation, our approach holds significant potential. We hope to stimulate further research and encourage exploring its applicability in other contexts, including political fact-checking and legal domains.