Stance Prediction and Claim Verification: An Arabic Perspective

TL;DR

This paper introduces an Arabic corpus for claim verification and stance prediction, develops baseline models using BERT, and analyzes the challenges of automatic claim verification relying solely on claim text.

Contribution

It provides a new Arabic dataset for claim verification and stance prediction, along with baseline models and analysis of the limitations of text-only claim verification.

Findings

BERT-based models achieved 76.7 F1 for stance prediction and 64.3 F1 for claim verification.

Pretraining features are useful for stance prediction but insufficient for claim verification without evidence.

Automatic claim verification faces significant challenges when only claim text is available.

Abstract

This work explores the application of textual entailment in news claim verification and stance prediction using a new corpus in Arabic. The publicly available corpus comes in two perspectives: a version consisting of 4,547 true and false claims and a version consisting of 3,786 pairs (claim, evidence). We describe the methodology for creating the corpus and the annotation process. Using the introduced corpus, we also develop two machine learning baselines for two proposed tasks: claim verification and stance prediction. Our best model utilizes pretraining (BERT) and achieves 76.7 F1 on the stance prediction task and 64.3 F1 on the claim verification task. Our preliminary experiments shed some light on the limits of automatic claim verification that relies on claims text only. Results hint that while the linguistic features and world knowledge learned during pretraining are useful for stance prediction, such learned representations from pretraining are insufficient for verifying claims without access to context or evidence.