SAFE: Similarity-Aware Multi-Modal Fake News Detection

TL;DR

SAFE is a multi-modal fake news detection method that leverages the relationship between textual and visual information to identify inconsistencies and improve detection accuracy.

Contribution

The paper introduces a novel similarity-aware approach that jointly learns textual, visual, and their relationship features for more effective fake news detection.

Findings

Outperforms existing methods on large-scale datasets

Effectively detects mismatches between text and images

Improves accuracy by exploiting cross-modal relationships

Abstract

Effective detection of fake news has recently attracted significant attention. Current studies have made significant contributions to predicting fake news with less focus on exploiting the relationship (similarity) between the textual and visual information in news articles. Attaching importance to such similarity helps identify fake news stories that, for example, attempt to use irrelevant images to attract readers' attention. In this work, we propose a $\mathsf{S}$imilarity-$\mathsf{A}$ware $\mathsf{F}$ak$\mathsf{E}$ news detection method ($\mathsf{SAFE}$) which investigates multi-modal (textual and visual) information of news articles. First, neural networks are adopted to separately extract textual and visual features for news representation. We further investigate the relationship between the extracted features across modalities. Such representations of news textual and visual information along with their relationship are jointly learned and used to predict fake news. The proposed method facilitates recognizing the falsity of news articles based on their text, images, or their "mismatches." We conduct extensive experiments on large-scale real-world data, which demonstrate the effectiveness of the proposed method.