ImageAttributionBench: How Far Are We from Generalizable Attribution?

TL;DR

This paper introduces ImageAttributionBench, a large, diverse dataset of synthetic images from advanced models, to evaluate and improve the robustness of image attribution methods against real-world challenges.

Contribution

The creation of a comprehensive, diverse dataset and benchmark for evaluating the generalization and robustness of image attribution techniques.

Findings

Current attribution methods perform poorly on degraded and semantically disjoint images.

Existing models lack robustness and generalization to unseen semantic content.

Abstract

The rapid advancement of generative AI has enabled the creation of highly realistic and diverse synthetic images, posing critical challenges for image provenance and misinformation detection. This underscores the urgent need for effective image attribution. However, existing attribution datasets are constrained by limited scale, outdated generation methods, and insufficient semantic diversity - hindering the development of robust and generalizable attribution models. To address these limitations, we introduce ImageAttributionBench, a comprehensive dataset comprising images synthesized by a wide array of advanced generative models with state-of-the-art (SOTA) architectures. Covering multiple real-world semantic domains, the dataset offers rich diversity and scale to support and accelerate progress in image attribution research. To simulate real-world attribution scenarios, we evaluate several SOTA attribution methods on ImageAttributionBench under two challenging settings: (1) training on a standard balanced split and testing on degraded images, and (2) training and testing on semantically disjoint splits. In both cases, current methods exhibit consistently poor performance, revealing significant limitations in their robustness and generalization to unseen semantic content. Our work provides a rigorous benchmark to facilitate the development and evaluation of future image attribution methods.