FantasyID: A dataset for detecting digital manipulations of ID-documents

TL;DR

FantasyID is a new, publicly available dataset designed to improve the detection of forged ID documents, challenging current algorithms with realistic, diverse, and complex manipulations to advance KYC security.

Contribution

The paper introduces FantasyID, a comprehensive dataset for ID forgery detection, including real-world-like IDs and simulated attacks, to facilitate development of more robust detection algorithms.

Findings

Current state-of-the-art algorithms struggle with FantasyID, especially at low false positive rates.

The dataset reveals significant challenges in forgery detection, with false negative rates near 50%.

FantasyID serves as a challenging benchmark for future detection algorithm improvements.

Abstract

Advancements in image generation led to the availability of easy-to-use tools for malicious actors to create forged images. These tools pose a serious threat to the widespread Know Your Customer (KYC) applications, requiring robust systems for detection of the forged Identity Documents (IDs). To facilitate the development of the detection algorithms, in this paper, we propose a novel publicly available (including commercial use) dataset, FantasyID, which mimics real-world IDs but without tampering with legal documents and, compared to previous public datasets, it does not contain generated faces or specimen watermarks. FantasyID contains ID cards with diverse design styles, languages, and faces of real people. To simulate a realistic KYC scenario, the cards from FantasyID were printed and captured with three different devices, constituting the bonafide class. We have emulated digital forgery/injection attacks that could be performed by a malicious actor to tamper the IDs using the existing generative tools. The current state-of-the-art forgery detection algorithms, such as TruFor, MMFusion, UniFD, and FatFormer, are challenged by FantasyID dataset. It especially evident, in the evaluation conditions close to practical, with the operational threshold set on validation set so that false positive rate is at 10%, leading to false negative rates close to 50% across the board on the test set. The evaluation experiments demonstrate that FantasyID dataset is complex enough to be used as an evaluation benchmark for detection algorithms.