Multimodal Matching-aware Co-attention Networks with Mutual Knowledge Distillation for Fake News Detection

TL;DR

This paper introduces a novel multimodal co-attention network with mutual knowledge distillation, leveraging image-text alignment to enhance fake news detection accuracy across multiple datasets.

Contribution

It proposes a matching-aware co-attention mechanism and a mutual knowledge distillation framework for improved multimodal fake news detection.

Findings

Achieves state-of-the-art performance on three benchmark datasets.

Effectively captures image-text alignment for better multimodal fusion.

Demonstrates the superiority of mutual knowledge distillation in fake news detection.

Abstract

Fake news often involves multimedia information such as text and image to mislead readers, proliferating and expanding its influence. Most existing fake news detection methods apply the co-attention mechanism to fuse multimodal features while ignoring the consistency of image and text in co-attention. In this paper, we propose multimodal matching-aware co-attention networks with mutual knowledge distillation for improving fake news detection. Specifically, we design an image-text matching-aware co-attention mechanism which captures the alignment of image and text for better multimodal fusion. The image-text matching representation can be obtained via a vision-language pre-trained model. Additionally, based on the designed image-text matching-aware co-attention mechanism, we propose to build two co-attention networks respectively centered on text and image for mutual knowledge distillation to improve fake news detection. Extensive experiments on three benchmark datasets demonstrate that our proposed model achieves state-of-the-art performance on multimodal fake news detection.