TSVer: A Benchmark for Fact Verification Against Time-Series Evidence

TL;DR

TSVer introduces a comprehensive benchmark dataset for fact verification involving temporal and numerical reasoning with real-world time-series evidence, addressing limitations of existing datasets.

Contribution

The paper presents TSVer, a new dataset with annotated claims and evidence, and establishes baseline performance for fact verification using time-series data.

Findings

Achieved an inter-annotator agreement of kappa=0.77 on verdicts.

State-of-the-art models like Gemini-2.5-Pro reach 63.57% accuracy on verdicts.

Models struggle with time-series evidence, indicating room for improvement.

Abstract

Reasoning over temporal and numerical data, such as time series, is a crucial aspect of fact-checking. While many systems have recently been developed to handle this form of evidence, their evaluation remains limited by existing datasets, which often lack structured evidence, provide insufficient justifications for verdicts, or rely on synthetic claims. In this paper, we introduce TSVer, a new benchmark dataset for fact verification focusing on temporal and numerical reasoning with time-series evidence. TSVer contains 304 real-world claims sourced from 41 fact-checking organizations and a curated database of 400 time series covering diverse domains. Each claim is annotated with time frames across all pertinent time series, along with a verdict and justifications reflecting how the evidence is used to reach the verdict. Using an LLM-assisted multi-step annotation process, we improve the quality of our annotations and achieve an inter-annotator agreement of kappa=0.77 on verdicts. We also develop a baseline for verifying claims against time-series evidence and show that even the state-of-the-art reasoning models like Gemini-2.5-Pro are challenged by time series, achieving a 63.57 accuracy score on verdicts and an Ev2R score of 47.36 on verdict justifications.