Supervised Contrastive Learning for Multimodal Unreliable News Detection in COVID-19 Pandemic

TL;DR

This paper introduces a BERT-based multimodal contrastive learning framework for detecting unreliable COVID-19 news by leveraging textual and visual information, outperforming baseline models.

Contribution

It presents a novel contrastive learning approach that jointly considers textual and visual data for improved unreliable news detection.

Findings

Outperforms baseline models on ReCOVery dataset

Effectively captures multimodal cues for credibility assessment

Demonstrates robustness in COVID-19 related news detection

Abstract

As the digital news industry becomes the main channel of information dissemination, the adverse impact of fake news is explosively magnified. The credibility of a news report should not be considered in isolation. Rather, previously published news articles on the similar event could be used to assess the credibility of a news report. Inspired by this, we propose a BERT-based multimodal unreliable news detection framework, which captures both textual and visual information from unreliable articles utilising the contrastive learning strategy. The contrastive learner interacts with the unreliable news classifier to push similar credible news (or similar unreliable news) closer while moving news articles with similar content but opposite credibility labels away from each other in the multimodal embedding space. Experimental results on a COVID-19 related dataset, ReCOVery, show that our model outperforms a number of competitive baseline in unreliable news detection.