GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image

TL;DR

This paper introduces the GenImage dataset, a large-scale benchmark with over one million AI-generated and real images, designed to improve detection of AI-generated images and evaluate detector robustness across diverse scenarios.

Contribution

The paper presents the GenImage dataset, the largest of its kind, including images from advanced generators, and proposes evaluation tasks for detector performance in real-world conditions.

Findings

Detectors trained on GenImage show strong generalization across different generators.

The dataset enables comprehensive evaluation of detection methods under various image degradations.

GenImage accelerates development of more robust AI-generated image detectors.

Abstract

The extraordinary ability of generative models to generate photographic images has intensified concerns about the spread of disinformation, thereby leading to the demand for detectors capable of distinguishing between AI-generated fake images and real images. However, the lack of large datasets containing images from the most advanced image generators poses an obstacle to the development of such detectors. In this paper, we introduce the GenImage dataset, which has the following advantages: 1) Plenty of Images, including over one million pairs of AI-generated fake images and collected real images. 2) Rich Image Content, encompassing a broad range of image classes. 3) State-of-the-art Generators, synthesizing images with advanced diffusion models and GANs. The aforementioned advantages allow the detectors trained on GenImage to undergo a thorough evaluation and demonstrate strong applicability to diverse images. We conduct a comprehensive analysis of the dataset and propose two tasks for evaluating the detection method in resembling real-world scenarios. The cross-generator image classification task measures the performance of a detector trained on one generator when tested on the others. The degraded image classification task assesses the capability of the detectors in handling degraded images such as low-resolution, blurred, and compressed images. With the GenImage dataset, researchers can effectively expedite the development and evaluation of superior AI-generated image detectors in comparison to prevailing methodologies.