Zero-Shot Fact Verification via Natural Logic and Large Language Models

TL;DR

This paper introduces a zero-shot fact verification method leveraging instruction-tuned large language models and natural logic, demonstrating superior generalization and transfer capabilities across artificial and real-world multilingual datasets.

Contribution

The paper proposes a novel zero-shot fact verification approach using instruction-tuned large language models with natural logic, reducing reliance on annotated training data.

Findings

Our method outperforms non-natural-logic systems by 8.96 accuracy points in zero-shot generalization.

It achieves better transfer performance across diverse datasets compared to models trained on natural logic.

The approach enhances explainability and domain transferability in fact verification.

Abstract

The recent development of fact verification systems with natural logic has enhanced their explainability by aligning claims with evidence through set-theoretic operators, providing faithful justifications. Despite these advancements, such systems often rely on a large amount of training data annotated with natural logic. To address this issue, we propose a zero-shot method that utilizes the generalization capabilities of instruction-tuned large language models. To comprehensively assess the zero-shot capabilities of our method and other fact verification systems, we evaluate all models on both artificial and real-world claims, including multilingual datasets. We also compare our method against other fact verification systems in two setups. First, in the zero-shot generalization setup, we demonstrate that our approach outperforms other systems that were not specifically trained on natural logic data, achieving an average accuracy improvement of 8.96 points over the best-performing baseline. Second, in the zero-shot transfer setup, we show that current systems trained on natural logic data do not generalize well to other domains, and our method outperforms these systems across all datasets with real-world claims.