Aggregating Pairwise Semantic Differences for Few-Shot Claim Veracity Classification

TL;DR

This paper introduces SEED, a vector-based method for few-shot claim veracity classification that aggregates semantic differences, outperforming baselines on FEVER and SCIFACT datasets.

Contribution

SEED is a novel approach that simulates class representative vectors to improve few-shot claim veracity classification performance.

Findings

SEED outperforms fine-tuned BERT/RoBERTa baselines.

SEED surpasses state-of-the-art perplexity-based methods.

Consistent improvements observed on FEVER and SCIFACT datasets.

Abstract

As part of an automated fact-checking pipeline, the claim veracity classification task consists in determining if a claim is supported by an associated piece of evidence. The complexity of gathering labelled claim-evidence pairs leads to a scarcity of datasets, particularly when dealing with new domains. In this paper, we introduce SEED, a novel vector-based method to few-shot claim veracity classification that aggregates pairwise semantic differences for claim-evidence pairs. We build on the hypothesis that we can simulate class representative vectors that capture average semantic differences for claim-evidence pairs in a class, which can then be used for classification of new instances. We compare the performance of our method with competitive baselines including fine-tuned BERT/RoBERTa models, as well as the state-of-the-art few-shot veracity classification method that leverages language model perplexity. Experiments conducted on the FEVER and SCIFACT datasets show consistent improvements over competitive baselines in few-shot settings. Our code is available.