KEN: Knowledge Augmentation and Emotion Guidance Network for Multimodal Fake News Detection

TL;DR

This paper introduces KEN, a multimodal fake news detection model that combines semantic understanding, knowledge augmentation, and emotion-guided learning to improve accuracy on social media misinformation.

Contribution

The paper presents a novel KEN framework that leverages LVLM for semantic understanding and incorporates emotion guidance for fine-grained fake news detection.

Findings

KEN outperforms existing methods on two real-world datasets.

Knowledge augmentation improves semantic comprehension of images and text.

Emotion-guided modeling enhances detection accuracy across different emotional news types.

Abstract

In recent years, the rampant spread of misinformation on social media has made accurate detection of multimodal fake news a critical research focus. However, previous research has not adequately understood the semantics of images, and models struggle to discern news authenticity with limited textual information. Meanwhile, treating all emotional types of news uniformly without tailored approaches further leads to performance degradation. Therefore, we propose a novel Knowledge Augmentation and Emotion Guidance Network (KEN). On the one hand, we effectively leverage LVLM's powerful semantic understanding and extensive world knowledge. For images, the generated captions provide a comprehensive understanding of image content and scenes, while for text, the retrieved evidence helps break the information silos caused by the closed and limited text and context. On the other hand, we consider inter-class differences between different emotional types of news through balanced learning, achieving fine-grained modeling of the relationship between emotional types and authenticity. Extensive experiments on two real-world datasets demonstrate the superiority of our KEN.