Multimodal Inverse Attention Network with Intrinsic Discriminant Feature Exploitation for Fake News Detection

TL;DR

This paper introduces MIAN, a novel multimodal fake news detection framework that leverages hierarchical intra-modal relationships and inverse attention to explicitly identify inconsistencies across modalities, significantly improving detection accuracy.

Contribution

The paper proposes a new framework, MIAN, which uniquely combines hierarchical intra-modal learning and inverse attention mechanisms for more effective multimodal fake news detection.

Findings

MIAN outperforms existing methods on benchmark datasets.

Hierarchical intra-modal learning enhances unimodal feature representations.

Inverse attention effectively captures cross-modal inconsistencies.

Abstract

Multimodal fake news detection has garnered significant attention due to its profound implications for social security. While existing approaches have contributed to understanding cross-modal consistency, they often fail to leverage modal-specific representations and explicit discrepant features. To address these limitations, we propose a Multimodal Inverse Attention Network (MIAN), a novel framework that explores intrinsic discriminative features based on news content to advance fake news detection. Specifically, MIAN introduces a hierarchical learning module that captures diverse intra-modal relationships through local-to-global and local-to-local interactions, thereby generating enhanced unimodal representations to improve the identification of fake news at the intra-modal level. Additionally, a cross-modal interaction module employs a co-attention mechanism to establish and model dependencies between the refined unimodal representations, facilitating seamless semantic integration across modalities. To explicitly extract inconsistency features, we propose an inverse attention mechanism that effectively highlights the conflicting patterns and semantic deviations introduced by fake news in both intra- and inter-modality. Extensive experiments on benchmark datasets demonstrate that MIAN significantly outperforms state-of-the-art methods, underscoring its pivotal contribution to advancing social security through enhanced multimodal fake news detection.