AIForge-Doc: A Benchmark for Detecting AI-Forged Tampering in Financial and Form Documents

TL;DR

AIForge-Doc introduces a new benchmark dataset for detecting AI-generated tampering in financial and form documents, revealing significant challenges for current detection methods due to the sophistication of AI inpainting techniques.

Contribution

This paper provides the first dedicated benchmark for diffusion-model-based inpainting in documents, with pixel-level annotations and evaluation of existing detectors, highlighting their limitations.

Findings

Existing detectors perform poorly on AI-forged documents.

AI-forged images are nearly indistinguishable to automated detectors and VLMs.

The benchmark exposes a critical gap in current document forgery detection capabilities.

Abstract

We present AIForge-Doc, the first dedicated benchmark targeting exclusively diffusion-model-based inpainting in financial and form documents with pixel-level annotation. Existing document forgery datasets rely on traditional digital editing tools (e.g., Adobe Photoshop, GIMP), creating a critical gap: state-of-the-art detectors are blind to the rapidly growing threat of AI-forged document fraud. AIForge-Doc addresses this gap by systematically forging numeric fields in real-world receipt and form images using two AI inpainting APIs -- Gemini 2.5 Flash Image and Ideogram v2 Edit -- yielding 4,061 forged images from four public document datasets (CORD, WildReceipt, SROIE, XFUND) across nine languages, annotated with pixel-precise tampered-region masks in DocTamper-compatible format. We benchmark three representative detectors -- TruFor, DocTamper, and a zero-shot GPT-4o judge -- and find that all existing methods degrade substantially: TruFor achieves AUC=0.751 (zero-shot, out-of-distribution) vs. AUC=0.96 on NIST16; DocTamper achieves AUC=0.563 vs. AUC=0.98 in-distribution, with pixel-level IoU=0.020; GPT-4o achieves only 0.509 -- essentially at chance -- confirming that AI-forged values are indistinguishable to automated detectors and VLMs. These results demonstrate that AIForge-Doc represents a qualitatively new and unsolved challenge for document forensics.