Can VLMs Detect and Localize Fine-Grained AI-Edited Images?

TL;DR

This paper introduces FragFake, a large-scale benchmark for detecting and localizing AI-generated image edits, and evaluates vision language models' performance, highlighting their strengths and limitations.

Contribution

It develops FragFake, a comprehensive benchmark dataset, and systematically studies the effectiveness of vision language models in fine-grained edited-image detection and localization.

Findings

Pretrained VLMs perform poorly on the task.

Fine-tuned models like Qwen2.5-VL achieve high accuracy.

Data balancing and training domain significantly impact performance.

Abstract

Fine-grained detection and localization of localized image edits is crucial for assessing content authenticity, especially as modern diffusion models and image editors can produce highly realistic manipulations. However, this problem faces three key challenges: (1) most AIGC detectors produce only a global real-or-fake label without indicating where edits occur; (2) traditional computer vision methods for edit localization typically rely on costly pixel-level annotations; and (3) there is no large-scale, modern benchmark specifically targeting edited-image detection. To address these gaps, we develop an automated data-generation pipeline and construct FragFake, a large-scale benchmark of AI-edited images spanning multiple source datasets, diverse editing models, and several common edit types. Building on FragFake, we are the first to systematically study vision language models (VLMs) for edited-image classification and edited-region localization. Our experiments show that pretrained VLMs, including GPT4o, perform poorly on this task, whereas fine-tuned models such as Qwen2.5-VL achieve high accuracy and substantially higher object precision across all settings. We further explore GRPO-based RLVR training, which yields modest metric gains while improving the interpretability of model outputs. Ablation and transfer analyses reveal how data balancing, training size, LoRA rank, and training domain affect performance, and highlight both the potential and the limitations of cross-editor and cross-dataset generalization. We anticipate that this work will establish a solid foundation to facilitate and inspire subsequent research endeavors in the domain of multimodal content authenticity.