Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models

TL;DR

This paper investigates how vision foundation models can be leveraged for training-free detection of AI-generated fake images, analyzing factors affecting detection performance and proposing methods to improve robustness across different image types.

Contribution

It provides a comprehensive analysis of detection performance across model backbones and perturbation types, introduces Contrastive Blur and MINDER to enhance detection accuracy, and offers insights into model robustness for deepfake detection.

Findings

Detection performance correlates with model robustness.

Self-supervised models yield more reliable representations.

Gaussian blur improves facial image detection.

Abstract

The rapid advancement of generative models has introduced serious risks, including deepfake techniques for facial synthesis and editing. Traditional approaches rely on training classifiers and enhancing generalizability through various feature extraction techniques. Meanwhile, training-free detection methods address issues like limited data and overfitting by directly leveraging statistical properties from vision foundation models to distinguish between real and fake images. The current leading training-free approach, RIGID, utilizes DINOv2 sensitivity to perturbations in image space for detecting fake images, with fake image embeddings exhibiting greater sensitivity than those of real images. This observation prompts us to investigate how detection performance varies across model backbones, perturbation types, and datasets. Our experiments reveal that detection performance is closely linked to model robustness, with self-supervised (SSL) models providing more reliable representations. While Gaussian noise effectively detects general objects, it performs worse on facial images, whereas Gaussian blur is more effective due to potential frequency artifacts. To further improve detection, we introduce Contrastive Blur, which enhances performance on facial images, and MINDER (MINimum distance DetEctoR), which addresses noise type bias, balancing performance across domains. Beyond performance gains, our work offers valuable insights for both the generative and detection communities, contributing to a deeper understanding of model robustness property utilized for deepfake detection.