SEER: Semantic Enhancement and Emotional Reasoning Network for Multimodal Fake News Detection

TL;DR

SEER is a novel multimodal fake news detection network that enhances semantic understanding and emotional reasoning, leveraging large multimodal models and emotional features to improve detection accuracy.

Contribution

The paper introduces a new SEER network that combines semantic enhancement with emotional reasoning, addressing gaps in existing multimodal fake news detection methods.

Findings

SEER outperforms state-of-the-art baselines on real-world datasets.

Semantic enhancement improves news understanding.

Emotional reasoning effectively captures fake news characteristics.

Abstract

Previous studies on multimodal fake news detection mainly focus on the alignment and integration of cross-modal features, as well as the application of text-image consistency. However, they overlook the semantic enhancement effects of large multimodal models and pay little attention to the emotional features of news. In addition, people find that fake news is more inclined to contain negative emotions than real ones. Therefore, we propose a novel Semantic Enhancement and Emotional Reasoning (SEER) Network for multimodal fake news detection. We generate summarized captions for image semantic understanding and utilize the products of large multimodal models for semantic enhancement. Inspired by the perceived relationship between news authenticity and emotional tendencies, we propose an expert emotional reasoning module that simulates real-life scenarios to optimize emotional features and infer the authenticity of news. Extensive experiments on two real-world datasets demonstrate the superiority of our SEER over state-of-the-art baselines.