Language Models as Fact Checkers?

TL;DR

This paper explores using language models as standalone fact checkers, leveraging their implicit knowledge to verify facts without external data, showing promising results on the FEVER benchmark.

Contribution

It is the first to evaluate language models as fact checkers in a closed-book setting, demonstrating their potential without relying on explicit knowledge bases.

Findings

Zero-shot LM outperforms random baseline on FEVER

Fine-tuned LM compares favorably with standard baselines

Method shows viability with room for improvement

Abstract

Recent work has suggested that language models (LMs) store both common-sense and factual knowledge learned from pre-training data. In this paper, we leverage this implicit knowledge to create an effective end-to-end fact checker using a solely a language model, without any external knowledge or explicit retrieval components. While previous work on extracting knowledge from LMs have focused on the task of open-domain question answering, to the best of our knowledge, this is the first work to examine the use of language models as fact checkers. In a closed-book setting, we show that our zero-shot LM approach outperforms a random baseline on the standard FEVER task, and that our fine-tuned LM compares favorably with standard baselines. Though we do not ultimately outperform methods which use explicit knowledge bases, we believe our exploration shows that this method is viable and has much room for exploration.