Explainable Automated Fact-Checking for Public Health Claims

TL;DR

This paper introduces an explainable fact-checking system for public health claims, creating a new dataset and evaluating veracity prediction and explanation quality to improve automated expertise-based fact-checking.

Contribution

It presents the first dataset and methodology for explainable fact-checking of expert-requiring claims, focusing on public health.

Findings

Training on in-domain data improves fact-checking accuracy.

Humans and algorithms agree on explanation coherence.

The dataset supports future research in specialized fact-checking.

Abstract

Fact-checking is the task of verifying the veracity of claims by assessing their assertions against credible evidence. The vast majority of fact-checking studies focus exclusively on political claims. Very little research explores fact-checking for other topics, specifically subject matters for which expertise is required. We present the first study of explainable fact-checking for claims which require specific expertise. For our case study we choose the setting of public health. To support this case study we construct a new dataset PUBHEALTH of 11.8K claims accompanied by journalist crafted, gold standard explanations (i.e., judgments) to support the fact-check labels for claims. We explore two tasks: veracity prediction and explanation generation. We also define and evaluate, with humans and computationally, three coherence properties of explanation quality. Our results indicate that, by training on in-domain data, gains can be made in explainable, automated fact-checking for claims which require specific expertise.