AMMeBa: A Large-Scale Survey and Dataset of Media-Based Misinformation In-The-Wild

TL;DR

This paper presents AMMeBa, a large-scale annotated dataset and survey of media-based misinformation, highlighting the rise of generative AI content and providing insights into misinformation modalities over two years.

Contribution

It introduces a comprehensive dataset of media-based misinformation annotations and offers a typology and analysis of misinformation types, especially focusing on AI-generated content.

Findings

Generative AI content in misinformation has increased recently.

Simple manipulation methods, especially context manipulations, dominate historically.

Media-based misinformation types and modalities have evolved over two years.

Abstract

The prevalence and harms of online misinformation is a perennial concern for internet platforms, institutions and society at large. Over time, information shared online has become more media-heavy and misinformation has readily adapted to these new modalities. The rise of generative AI-based tools, which provide widely-accessible methods for synthesizing realistic audio, images, video and human-like text, have amplified these concerns. Despite intense public interest and significant press coverage, quantitative information on the prevalence and modality of media-based misinformation remains scarce. Here, we present the results of a two-year study using human raters to annotate online media-based misinformation, mostly focusing on images, based on claims assessed in a large sample of publicly-accessible fact checks with the ClaimReview markup. We present an image typology, designed to capture aspects of the image and manipulation relevant to the image's role in the misinformation claim. We visualize the distribution of these types over time. We show the rise of generative AI-based content in misinformation claims, and that its commonality is a relatively recent phenomenon, occurring significantly after heavy press coverage. We also show "simple" methods dominated historically, particularly context manipulations, and continued to hold a majority as of the end of data collection in November 2023. The dataset, Annotated Misinformation, Media-Based (AMMeBa), is publicly-available, and we hope that these data will serve as both a means of evaluating mitigation methods in a realistic setting and as a first-of-its-kind census of the types and modalities of online misinformation.