BERT for Evidence Retrieval and Claim Verification

TL;DR

This paper explores using BERT models for evidence retrieval and claim verification in fact-checking, achieving state-of-the-art recall and high leaderboard scores in the FEVER challenge.

Contribution

The paper introduces a two-BERT model pipeline for evidence retrieval and claim verification, with novel training strategies including pointwise and pairwise loss functions and hard negative mining.

Findings

Achieved 87.1% recall for top five evidence retrieval

Scored 69.7 FEVER score, second place in leaderboard

Demonstrated effectiveness of BERT in fact verification tasks

Abstract

Motivated by the promising performance of pre-trained language models, we investigate BERT in an evidence retrieval and claim verification pipeline for the FEVER fact extraction and verification challenge. To this end, we propose to use two BERT models, one for retrieving potential evidence sentences supporting or rejecting claims, and another for verifying claims based on the predicted evidence sets. To train the BERT retrieval system, we use pointwise and pairwise loss functions, and examine the effect of hard negative mining. A second BERT model is trained to classify the samples as supported, refuted, and not enough information. Our system achieves a new state of the art recall of 87.1 for retrieving top five sentences out of the FEVER documents consisting of 50K Wikipedia pages, and scores second in the official leaderboard with the FEVER score of 69.7.