Self-Adaptive Paraphrasing and Preference Learning for Improved Claim Verifiability

TL;DR

This paper introduces a self-adaptive, data-label-free method to generate claim paraphrases from social media content, improving fact-checking accuracy by optimizing claim verifiability through preference learning.

Contribution

It proposes a novel self-adaptive approach that leverages preference optimization and a black-box fact-checker to enhance claim extraction without labeled data.

Findings

Outperforms baselines in claim verifiability

Improves fact-checking accuracy on social media claims

Effective for refuted claims

Abstract

In fact-checking, structure and phrasing of claims critically influence a model's ability to predict verdicts accurately. Social media content in particular rarely serves as optimal input for verification systems, which necessitates pre-processing to extract the claim from noisy context before fact checking. Prior work suggests extracting a claim representation that humans find to be checkworthy and verifiable. This has two limitations: (1) the format may not be optimal for a fact-checking model, and (2), it requires annotated data to learn the extraction task from. We address both issues and propose a method to extract claims that is not reliant on labeled training data. Instead, our self-adaptive approach only requires a black-box fact checking model and a generative language model (LM). Given a tweet, we iteratively optimize the LM to generate a claim paraphrase that increases the performance of a fact checking model. By learning from preference pairs, we align the LM to the fact checker using direct preference optimization. We show that this novel setup extracts a claim paraphrase that is more verifiable than their original social media formulations, and is on par with competitive baselines. For refuted claims, our method consistently outperforms all baselines.