A Multi-Agent Approach for Claim Verification from Tabular Data Documents

TL;DR

This paper introduces MACE, a multi-agent framework for claim verification from tabular data that achieves state-of-the-art results with smaller models and offers transparent reasoning without extensive fine-tuning.

Contribution

The paper proposes a novel multi-agent approach using zero-shot Chain-of-Thought prompting for interpretable claim verification, outperforming larger models.

Findings

MACE achieves SOTA performance on two datasets.

MACE performs comparably to top models on two other datasets.

Smaller models (27-92B parameters) reach high accuracy with transparent reasoning.

Abstract

We present a novel approach for claim verification from tabular data documents. Recent LLM-based approaches either employ complex pretraining/fine-tuning or decompose verification into subtasks, often lacking comprehensive explanations and generalizability. To address these limitations, we propose a Multi-Agentic framework for Claim verification (MACE) consisting of three specialized agents: Planner, Executor, and Verifier. Instead of elaborate finetuning, each agent employs a zero-shot Chain-of-Thought setup to perform its tasks. MACE produces interpretable verification traces, with the Planner generating explicit reasoning strategies, the Executor providing detailed computation steps, and the Verifier validating the logic. Experiments demonstrate that MACE achieves state-of-the-art (SOTA) performance on two datasets and performs on par with the best models on two others, while achieving 80--100\% of best performance with substantially smaller models: 27--92B parameters versus 235B. This combination of competitive performance, memory efficiency, and transparent reasoning highlights our framework's effectiveness.