Weakly Supervised Training for Hologram Verification in Identity Documents

TL;DR

This paper introduces a weakly supervised method for verifying holograms in identity documents using smartphone videos, achieving state-of-the-art results and addressing photo replacement attacks with minimal supervision.

Contribution

It presents a novel weakly supervised training approach for hologram verification that works with limited labeled data and can detect photo replacement attacks.

Findings

Achieved new leading performance on MIDV-HOLO dataset.

Maintains high recall on attack samples from MIDV-2020.

First method to effectively address photo replacement attacks.

Abstract

We propose a method to remotely verify the authenticity of Optically Variable Devices (OVDs), often referred to as ``holograms'', in identity documents. Our method processes video clips captured with smartphones under common lighting conditions, and is evaluated on two public datasets: MIDV-HOLO and MIDV-2020. Thanks to a weakly-supervised training, we optimize a feature extraction and decision pipeline which achieves a new leading performance on MIDV-HOLO, while maintaining a high recall on documents from MIDV-2020 used as attack samples. It is also the first method, to date, to effectively address the photo replacement attack task, and can be trained on either genuine samples, attack samples, or both for increased performance. By enabling to verify OVD shapes and dynamics with very little supervision, this work opens the way towards the use of massive amounts of unlabeled data to build robust remote identity document verification systems on commodity smartphones. Code is available at https://github.com/EPITAResearchLab/pouliquen.24.icdar