Multimodal Dual Emotion with Fusion of Visual Sentiment for Rumor Detection

TL;DR

This paper introduces a novel multimodal approach combining visual and textual emotions to improve rumor detection accuracy, emphasizing the importance of image sentiment analysis which was previously overlooked.

Contribution

It proposes the first use of visual emotion features in rumor detection, enhancing existing models with multimodal emotional cues.

Findings

Visual emotion features outperform traditional sentiment features.

The proposed method improves rumor detection accuracy.

Experiments on real datasets validate the effectiveness of visual emotion integration.

Abstract

In recent years, rumors have had a devastating impact on society, making rumor detection a significant challenge. However, the studies on rumor detection ignore the intense emotions of images in the rumor content. This paper verifies that the image emotion improves the rumor detection efficiency. A Multimodal Dual Emotion feature in rumor detection, which consists of visual and textual emotions, is proposed. To the best of our knowledge, this is the first study which uses visual emotion in rumor detection. The experiments on real datasets verify that the proposed features outperform the state-of-the-art sentiment features, and can be extended in rumor detectors while improving their performance.