TwoHead-SwinFPN: A Unified DL Architecture for Synthetic Manipulation, Detection and Localization in Identity Documents

TL;DR

This paper introduces TwoHead-SwinFPN, a deep learning model that effectively detects and localizes synthetic manipulations in identity documents, enhancing security against AI-generated forgeries.

Contribution

The paper presents a novel unified architecture combining Swin Transformer, FPN, and UNet components with multi-task learning for manipulation detection and localization in ID documents.

Findings

Achieves 84.31% accuracy and 90.78% AUC in classification

Attains 57.24% mean Dice score in localization

Demonstrates robustness across multiple languages and devices

Abstract

The proliferation of sophisticated generative AI models has significantly escalated the threat of synthetic manipulations in identity documents, particularly through face swapping and text inpainting attacks. This paper presents TwoHead-SwinFPN, a unified deep learning architecture that simultaneously performs binary classification and precise localization of manipulated regions in ID documents. Our approach integrates a Swin Transformer backbone with Feature Pyramid Network (FPN) and UNet-style decoder, enhanced with Convolutional Block Attention Module (CBAM) for improved feature representation. The model employs a dual-head architecture for joint optimization of detection and segmentation tasks, utilizing uncertainty-weighted multi-task learning. Extensive experiments on the FantasyIDiap dataset demonstrate superior performance with 84.31\% accuracy, 90.78\% AUC for classification, and 57.24\% mean Dice score for localization. The proposed method achieves an F1-score of 88.61\% for binary classification while maintaining computational efficiency suitable for real-world deployment through FastAPI implementation. Our comprehensive evaluation includes ablation studies, cross-device generalization analysis, and detailed performance assessment across 10 languages and 3 acquisition devices.