Integrating Stance Detection and Fact Checking in a Unified Corpus

TL;DR

This paper introduces a unified Arabic fact checking corpus that integrates document retrieval, stance detection, source credibility, and rationale extraction to improve fact checking systems.

Contribution

It presents the first Arabic fact checking corpus with integrated annotations for multiple interdependent tasks, enabling more comprehensive fact checking models.

Findings

First Arabic corpus with integrated fact checking tasks

Supports interdependent task annotations in a single dataset

Facilitates development of more explainable fact checking systems

Abstract

A reasonable approach for fact checking a claim involves retrieving potentially relevant documents from different sources (e.g., news websites, social media, etc.), determining the stance of each document with respect to the claim, and finally making a prediction about the claim's factuality by aggregating the strength of the stances, while taking the reliability of the source into account. Moreover, a fact checking system should be able to explain its decision by providing relevant extracts (rationales) from the documents. Yet, this setup is not directly supported by existing datasets, which treat fact checking, document retrieval, source credibility, stance detection and rationale extraction as independent tasks. In this paper, we support the interdependencies between these tasks as annotations in the same corpus. We implement this setup on an Arabic fact checking corpus, the first of its kind.