KG-CRAFT: Knowledge Graph-based Contrastive Reasoning with LLMs for Enhancing Automated Fact-checking

TL;DR

KG-CRAFT enhances automated fact-checking by integrating knowledge graph-guided contrastive questions with LLMs, significantly improving claim verification accuracy through evidence synthesis and reasoning.

Contribution

This work introduces a novel knowledge graph-based contrastive reasoning framework that augments LLMs for more accurate automated claim verification.

Findings

Achieves state-of-the-art performance on LIAR-RAW and RAWFC datasets.

Effectively leverages knowledge graphs for contrastive question generation.

Improves LLM-based fact-checking accuracy through structured reasoning.

Abstract

Claim verification is a core component of automated fact-checking systems, aimed at determining the truthfulness of a statement by assessing it against reliable evidence sources such as documents or knowledge bases. This work presents KG-CRAFT, a method that improves automatic claim verification by leveraging large language models (LLMs) augmented with contrastive questions grounded in a knowledge graph. KG-CRAFT first constructs a knowledge graph from claims and associated reports, then formulates contextually relevant contrastive questions based on the knowledge graph structure. These questions guide the distillation of evidence-based reports, which are synthesised into a concise summary that is used for veracity assessment by LLMs. Extensive evaluations on two real-world datasets (LIAR-RAW and RAWFC) demonstrate that our method achieves a new state-of-the-art in predictive performance. Comprehensive analyses validate in detail the effectiveness of our knowledge graph-based contrastive reasoning approach in improving LLMs' fact-checking capabilities.