TGIF: Text-Guided Inpainting Forgery Dataset

TL;DR

This paper introduces the TGIF dataset, a large collection of images for training and evaluating forgery detection methods, highlighting the limitations of current techniques against modern AI-generated inpainting manipulations.

Contribution

The paper presents the TGIF dataset with 75k forged images from popular AI models, and benchmarks existing detection methods, revealing their weaknesses against modern inpainting forgeries.

Findings

Traditional IFL methods detect spliced images but not regenerated inpainted images.

Traditional SID methods detect fake images but cannot localize inpainted areas.

Current detectors are ineffective against modern compression and AI-generated manipulations.

Abstract

Digital image manipulation has become increasingly accessible and realistic with the advent of generative AI technologies. Recent developments allow for text-guided inpainting, making sophisticated image edits possible with minimal effort. This poses new challenges for digital media forensics. For example, diffusion model-based approaches could either splice the inpainted region into the original image, or regenerate the entire image. In the latter case, traditional image forgery localization (IFL) methods typically fail. This paper introduces the Text-Guided Inpainting Forgery (TGIF) dataset, a comprehensive collection of images designed to support the training and evaluation of image forgery localization and synthetic image detection (SID) methods. The TGIF dataset includes approximately 75k forged images, originating from popular open-source and commercial methods, namely SD2, SDXL, and Adobe Firefly. We benchmark several state-of-the-art IFL and SID methods on TGIF. Whereas traditional IFL methods can detect spliced images, they fail to detect regenerated inpainted images. Moreover, traditional SID may detect the regenerated inpainted images to be fake, but cannot localize the inpainted area. Finally, both IFL and SID methods fail when exposed to stronger compression, while they are less robust to modern compression algorithms, such as WEBP. In conclusion, this work demonstrates the inefficiency of state-of-the-art detectors on local manipulations performed by modern generative approaches, and aspires to help with the development of more capable IFL and SID methods. The dataset and code can be downloaded at https://github.com/IDLabMedia/tgif-dataset.