Evidence-based Interpretable Open-domain Fact-checking with Large Language Models

TL;DR

This paper presents OE-Fact, an open-domain fact-checking system leveraging large language models to retrieve, verify, and explain claims in real-time, outperforming baselines on the FEVER dataset.

Contribution

The work introduces a novel open-domain fact-checking framework using LLMs for evidence retrieval, verification, and explanation generation, adapting the traditional three-module approach.

Findings

OE-Fact outperforms baseline systems in accuracy.

The system provides real-time, concise explanations.

Effective evidence retrieval from open websites.

Abstract

Universal fact-checking systems for real-world claims face significant challenges in gathering valid and sufficient real-time evidence and making reasoned decisions. In this work, we introduce the Open-domain Explainable Fact-checking (OE-Fact) system for claim-checking in real-world scenarios. The OE-Fact system can leverage the powerful understanding and reasoning capabilities of large language models (LLMs) to validate claims and generate causal explanations for fact-checking decisions. To adapt the traditional three-module fact-checking framework to the open domain setting, we first retrieve claim-related information as relevant evidence from open websites. After that, we retain the evidence relevant to the claim through LLM and similarity calculation for subsequent verification. We evaluate the performance of our adapted three-module OE-Fact system on the Fact Extraction and Verification (FEVER) dataset. Experimental results show that our OE-Fact system outperforms general fact-checking baseline systems in both closed- and open-domain scenarios, ensuring stable and accurate verdicts while providing concise and convincing real-time explanations for fact-checking decisions.