Insta-VAX: A Multimodal Benchmark for Anti-Vaccine and Misinformation Posts Detection on Social Media

TL;DR

Insta-VAX introduces a large multimodal dataset of Instagram vaccine-related posts and benchmarks models for detecting anti-vaccine content and misinformation, highlighting the importance of combining text and visual analysis.

Contribution

The paper presents a new multimodal dataset and evaluates state-of-the-art models for vaccine misinformation detection, emphasizing the need for improved visual context understanding.

Findings

Multimodal models outperform unimodal models in classification accuracy.

Current models still struggle with visual context and external knowledge integration.

The dataset supports social and public health efforts in misinformation monitoring.

Abstract

Sharing of anti-vaccine posts on social media, including misinformation posts, has been shown to create confusion and reduce the publics confidence in vaccines, leading to vaccine hesitancy and resistance. Recent years have witnessed the fast rise of such anti-vaccine posts in a variety of linguistic and visual forms in online networks, posing a great challenge for effective content moderation and tracking. Extending previous work on leveraging textual information to understand vaccine information, this paper presents Insta-VAX, a new multi-modal dataset consisting of a sample of 64,957 Instagram posts related to human vaccines. We applied a crowdsourced annotation procedure verified by two trained expert judges to this dataset. We then bench-marked several state-of-the-art NLP and computer vision classifiers to detect whether the posts show anti-vaccine attitude and whether they contain misinformation. Extensive experiments and analyses demonstrate the multimodal models can classify the posts more accurately than the uni-modal models, but still need improvement especially on visual context understanding and external knowledge cooperation. The dataset and classifiers contribute to monitoring and tracking of vaccine discussions for social scientific and public health efforts in combating the problem of vaccine misinformation.