Thucy: An LLM-based Multi-Agent System for Claim Verification across Relational Databases

TL;DR

Thucy is a novel multi-agent system leveraging LLMs to verify claims across multiple relational databases, providing transparent evidence and surpassing previous benchmarks in accuracy.

Contribution

It introduces the first cross-database, multi-table claim verification system that is data-source agnostic and offers explicit SQL-based evidence for its verdicts.

Findings

Achieved 94.3% accuracy on TabFact, surpassing previous state-of-the-art by 5.6%.

Provides transparent SQL queries supporting each verification decision.

Operates effectively across multiple relational databases without prior data source knowledge.

Abstract

In today's age, it is becoming increasingly difficult to decipher truth from lies. Every day, politicians, media outlets, and public figures make conflicting claims -- often about topics that can, in principle, be verified against structured data. For instance, statements about crime rates, economic growth or healthcare can all be verified against official public records and structured datasets. Building a system that can automatically do that would have sounded like science fiction just a few years ago. Yet, with the extraordinary progress in LLMs and agentic AI, this is now within reach. Still, there remains a striking gap between what is technically possible and what is being demonstrated by recent work. Most existing verification systems operate only on small, single-table databases -- typically a few hundred rows -- that conveniently fit within an LLM's context window. In this paper we report our progress on Thucy, the first cross-database, cross-table multi-agent claim verification system that also provides concrete evidence for each verification verdict. Thucy remains completely agnostic to the underlying data sources before deployment and must therefore autonomously discover, inspect, and reason over all available relational databases to verify claims. Importantly, Thucy also reports the exact SQL queries that support its verdict (whether the claim is accurate or not) offering full transparency to expert users familiar with SQL. When evaluated on the TabFact dataset -- the standard benchmark for fact verification over structured data -- Thucy surpasses the previous state of the art by 5.6 percentage points in accuracy (94.3% vs. 88.7%).