Improving Fake News Detection by Using an Entity-enhanced Framework to Fuse Diverse Multimodal Clues

TL;DR

This paper introduces an entity-enhanced multimodal framework that effectively models high-level semantic correlations between text and images, significantly improving fake news detection accuracy.

Contribution

It proposes a novel framework that captures entity inconsistency, mutual enhancement, and text complementation in multimodal fake news detection, leveraging visual entities for better semantic understanding.

Findings

Outperforms existing models in fake news detection accuracy

Effectively models three key cross-modal correlations

Demonstrates robustness across diverse datasets

Abstract

Recently, fake news with text and images have achieved more effective diffusion than text-only fake news, raising a severe issue of multimodal fake news detection. Current studies on this issue have made significant contributions to developing multimodal models, but they are defective in modeling the multimodal content sufficiently. Most of them only preliminarily model the basic semantics of the images as a supplement to the text, which limits their performance on detection. In this paper, we find three valuable text-image correlations in multimodal fake news: entity inconsistency, mutual enhancement, and text complementation. To effectively capture these multimodal clues, we innovatively extract visual entities (such as celebrities and landmarks) to understand the news-related high-level semantics of images, and then model the multimodal entity inconsistency and mutual enhancement with the help of visual entities. Moreover, we extract the embedded text in images as the complementation of the original text. All things considered, we propose a novel entity-enhanced multimodal fusion framework, which simultaneously models three cross-modal correlations to detect diverse multimodal fake news. Extensive experiments demonstrate the superiority of our model compared to the state of the art.