The Synthetic Media Shift: Tracking the Rise, Virality, and Detectability of AI-Generated Multimodal Misinformation

TL;DR

This paper introduces CONVEX, a large multimodal misinformation dataset, and analyzes how AI-generated content spreads, is detected, and impacts online discourse, emphasizing the need for adaptive detection strategies.

Contribution

The study provides a comprehensive dataset and analysis of multimodal misinformation dynamics, revealing challenges in detecting evolving AI-generated media.

Findings

AI-generated content achieves disproportionate virality

Passive engagement drives spread more than active discourse

Detection models' performance declines as generative models evolve

Abstract

As generative AI advances, the distinction between authentic and synthetic media is increasingly blurred, challenging the integrity of online information. In this study, we present CONVEX, a large-scale dataset of multimodal misinformation involving miscaptioned, edited, and AI-generated visual content, comprising over 150K multimodal posts with associated notes and engagement metrics from X's Community Notes. We analyze how multimodal misinformation evolves in terms of virality, engagement, and consensus dynamics, with a focus on synthetic media. Our results show that while AI-generated content achieves disproportionate virality, its spread is driven primarily by passive engagement rather than active discourse. Despite slower initial reporting, AI-generated content reaches community consensus more quickly once flagged. Moreover, our evaluation of specialized detectors and vision-language models reveals a consistent decline in performance over time in distinguishing synthetic from authentic images as generative models evolve. These findings highlight the need for continuous monitoring and adaptive strategies in the rapidly evolving digital information environment.