FACT-GPT: Fact-Checking Augmentation via Claim Matching with LLMs

TL;DR

FACT-GPT leverages specialized large language models to automate claim matching in fact-checking, effectively identifying related, contradictory, or irrelevant social media content, thus aiding in combating misinformation.

Contribution

Introduces FACT-GPT, a novel LLM-based system trained on synthetic data to automate claim matching in fact-checking, matching larger models' accuracy and supporting misinformation mitigation.

Findings

Specialized LLMs match larger models in claim matching accuracy

FACT-GPT effectively identifies related, contradictory, or irrelevant claims

Automated claim matching supports efficient fact-checking processes

Abstract

Our society is facing rampant misinformation harming public health and trust. To address the societal challenge, we introduce FACT-GPT, a system leveraging Large Language Models (LLMs) to automate the claim matching stage of fact-checking. FACT-GPT, trained on a synthetic dataset, identifies social media content that aligns with, contradicts, or is irrelevant to previously debunked claims. Our evaluation shows that our specialized LLMs can match the accuracy of larger models in identifying related claims, closely mirroring human judgment. This research provides an automated solution for efficient claim matching, demonstrates the potential of LLMs in supporting fact-checkers, and offers valuable resources for further research in the field.