Ask to Know More: Generating Counterfactual Explanations for Fake Claims

TL;DR

This paper introduces a method for generating counterfactual explanations for fake news detection, aiming to improve public understanding of why news is classified as fake by leveraging question answering and contradiction detection.

Contribution

It proposes a novel approach to generate counterfactual explanations for fake news using contradiction reasoning and question answering, enhancing interpretability of fact checking systems.

Findings

Outperforms state-of-the-art explanation methods

Generates helpful explanations for fake news detection

Improves user understanding of fact checking predictions

Abstract

Automated fact checking systems have been proposed that quickly provide veracity prediction at scale to mitigate the negative influence of fake news on people and on public opinion. However, most studies focus on veracity classifiers of those systems, which merely predict the truthfulness of news articles. We posit that effective fact checking also relies on people's understanding of the predictions. In this paper, we propose elucidating fact checking predictions using counterfactual explanations to help people understand why a specific piece of news was identified as fake. In this work, generating counterfactual explanations for fake news involves three steps: asking good questions, finding contradictions, and reasoning appropriately. We frame this research question as contradicted entailment reasoning through question answering (QA). We first ask questions towards the false claim and retrieve potential answers from the relevant evidence documents. Then, we identify the most contradictory answer to the false claim by use of an entailment classifier. Finally, a counterfactual explanation is created using a matched QA pair with three different counterfactual explanation forms. Experiments are conducted on the FEVER dataset for both system and human evaluations. Results suggest that the proposed approach generates the most helpful explanations compared to state-of-the-art methods.