Are CLIP features all you need for Universal Synthetic Image Origin Attribution?

TL;DR

This paper introduces a new method using features from large pre-trained models to accurately attribute the source of synthetic images, including unseen models, outperforming existing techniques especially with limited data.

Contribution

It presents a simple, effective framework leveraging foundation model features for open-set synthetic image attribution across various generative models.

Findings

High attribution accuracy across diverse models

Effective in low-data scenarios

Better generalization to unseen architectures

Abstract

The steady improvement of Diffusion Models for visual synthesis has given rise to many new and interesting use cases of synthetic images but also has raised concerns about their potential abuse, which poses significant societal threats. To address this, fake images need to be detected and attributed to their source model, and given the frequent release of new generators, realistic applications need to consider an Open-Set scenario where some models are unseen at training time. Existing forensic techniques are either limited to Closed-Set settings or to GAN-generated images, relying on fragile frequency-based "fingerprint" features. By contrast, we propose a simple yet effective framework that incorporates features from large pre-trained foundation models to perform Open-Set origin attribution of synthetic images produced by various generative models, including Diffusion Models. We show that our method leads to remarkable attribution performance, even in the low-data regime, exceeding the performance of existing methods and generalizes better on images obtained from a diverse set of architectures. We make the code publicly available at: https://github.com/ciodar/UniversalAttribution.