TabVer: Tabular Fact Verification with Natural Logic

TL;DR

This paper introduces TabVer, a natural logic-based method for tabular fact verification that integrates arithmetic reasoning, achieving state-of-the-art accuracy on FEVEROUS and competitive results on TabFact without additional training.

Contribution

It presents a novel set-theoretic interpretation of numerals and arithmetic in natural logic, enabling effective reasoning over tabular data with large language models.

Findings

Achieves 71.4% accuracy on FEVEROUS, surpassing previous models by 3.4 points.

Maintains competitive performance on TabFact without further training.

Demonstrates the effectiveness of natural logic with arithmetic in tabular fact verification.

Abstract

Fact verification on tabular evidence incentivises the use of symbolic reasoning models where a logical form is constructed (e.g. a LISP-style program), providing greater verifiability than fully neural approaches. However, these systems typically rely on well-formed tables, restricting their use in many scenarios. An emerging symbolic reasoning paradigm for textual evidence focuses on natural logic inference, which constructs proofs by modelling set-theoretic relations between a claim and its evidence in natural language. This approach provides flexibility and transparency but is less compatible with tabular evidence since the relations do not extend to arithmetic functions. We propose a set-theoretic interpretation of numerals and arithmetic functions in the context of natural logic, enabling the integration of arithmetic expressions in deterministic proofs. We leverage large language models to generate arithmetic expressions by generating questions about salient parts of a claim which are answered by executing appropriate functions on tables. In a few-shot setting on FEVEROUS, we achieve an accuracy of 71.4, outperforming both fully neural and symbolic reasoning models by 3.4 points. When evaluated on TabFact without any further training, our method remains competitive with an accuracy lead of 0.5 points.