Verifying the Verifiers: Unveiling Pitfalls and Potentials in Fact Verifiers

TL;DR

This paper evaluates various large language models for fact verification, highlighting dataset issues, the effectiveness of frontier LLMs with few-shot learning, and the potential of fine-tuned smaller models with synthetic data for improved accuracy.

Contribution

It uncovers dataset annotation problems, emphasizes the importance of frontier LLMs in fact verification, and proposes training smaller models with synthetic data to enhance reasoning capabilities.

Findings

16% of dataset labels are ambiguous or incorrect, affecting model rankings.

Frontier LLMs with few-shot learning outperform many baselines.

Synthetic multi-hop reasoning data improves small model performance.

Abstract

Fact verification is essential for ensuring the reliability of LLM applications. In this study, we evaluate 12 pre-trained LLMs and one specialized fact-verifier, including frontier LLMs and open-weight reasoning LLMs, using a collection of examples from 14 fact-checking benchmarks. We share three findings intended to guide future development of more robust fact verifiers. First, we highlight the importance of addressing annotation errors and ambiguity in datasets, demonstrating that approximately 16\% of ambiguous or incorrectly labeled data substantially influences model rankings. Neglecting this issue may result in misleading conclusions during comparative evaluations, and we suggest using a systematic pipeline utilizing LLM-as-a-judge to help identify these issues at scale. Second, we discover that frontier LLMs with few-shot in-context examples, often overlooked in previous works, achieve top-tier performance. We therefore recommend that future studies include comparisons with these simple yet highly effective baselines. Lastly, despite their effectiveness, frontier LLMs incur substantial costs, motivating the development of small, fine-tuned fact verifiers. We show that these small models still have room for improvement, particularly on instances that require complex reasoning. Encouragingly, we demonstrate that augmenting training with synthetic multi-hop reasoning data significantly enhances their capabilities in such instances. We release our code, model, and dataset at https://github.com/just1nseo/verifying-the-verifiers.