Towards Universal Fake Image Detectors that Generalize Across Generative Models

TL;DR

This paper introduces a new approach for detecting fake images that generalizes across different generative models by using feature spaces from pretrained models, outperforming existing methods.

Contribution

It proposes a paradigm shift from training classifiers to using feature space similarity measures for fake image detection, enhancing generalization to unseen models.

Findings

Nearest neighbor classification in pretrained feature space outperforms state-of-the-art methods.

The approach generalizes well to unseen diffusion and autoregressive models.

Significant improvements in accuracy and mAP on diverse generative models.

Abstract

With generative models proliferating at a rapid rate, there is a growing need for general purpose fake image detectors. In this work, we first show that the existing paradigm, which consists of training a deep network for real-vs-fake classification, fails to detect fake images from newer breeds of generative models when trained to detect GAN fake images. Upon analysis, we find that the resulting classifier is asymmetrically tuned to detect patterns that make an image fake. The real class becomes a sink class holding anything that is not fake, including generated images from models not accessible during training. Building upon this discovery, we propose to perform real-vs-fake classification without learning; i.e., using a feature space not explicitly trained to distinguish real from fake images. We use nearest neighbor and linear probing as instantiations of this idea. When given access to the feature space of a large pretrained vision-language model, the very simple baseline of nearest neighbor classification has surprisingly good generalization ability in detecting fake images from a wide variety of generative models; e.g., it improves upon the SoTA by +15.07 mAP and +25.90% acc when tested on unseen diffusion and autoregressive models.