Linked Credibility Reviews for Explainable Misinformation Detection

TL;DR

This paper introduces a novel architecture using Credibility Reviews to build explainable, domain-independent misinformation detection networks that leverage existing credibility signals and semantic analysis, achieving state-of-the-art results without fine-tuning.

Contribution

The paper proposes a new Credibility Review-based architecture for misinformation detection that enhances explainability, extensibility, and domain-independence compared to existing deep learning methods.

Findings

Achieves state-of-the-art on the Clef'18 CheckThat! Factuality task.

Demonstrates advantages in explainability and transparency.

Operates effectively without fine-tuning deep models.

Abstract

In recent years, misinformation on the Web has become increasingly rampant. The research community has responded by proposing systems and challenges, which are beginning to be useful for (various subtasks of) detecting misinformation. However, most proposed systems are based on deep learning techniques which are fine-tuned to specific domains, are difficult to interpret and produce results which are not machine readable. This limits their applicability and adoption as they can only be used by a select expert audience in very specific settings. In this paper we propose an architecture based on a core concept of Credibility Reviews (CRs) that can be used to build networks of distributed bots that collaborate for misinformation detection. The CRs serve as building blocks to compose graphs of (i) web content, (ii) existing credibility signals --fact-checked claims and reputation reviews of websites--, and (iii) automatically computed reviews. We implement this architecture on top of lightweight extensions to Schema.org and services providing generic NLP tasks for semantic similarity and stance detection. Evaluations on existing datasets of social-media posts, fake news and political speeches demonstrates several advantages over existing systems: extensibility, domain-independence, composability, explainability and transparency via provenance. Furthermore, we obtain competitive results without requiring finetuning and establish a new state of the art on the Clef'18 CheckThat! Factuality task.