Towards Zero-Shot Differential Morphing Attack Detection with Multimodal Large Language Models

TL;DR

This paper explores the novel use of multimodal large language models for differential morphing attack detection in biometric data, employing Chain-of-Thought prompts to improve interpretability and benchmarking two models on real-world passport data.

Contribution

It is the first to apply multimodal LLMs to D-MAD with real biometric data and introduces CoT prompt engineering for better reliability and explainability.

Findings

ChatGPT-4o outperforms Gemini in detection accuracy.

Both models struggle under challenging conditions.

Gemini provides more consistent explanations.

Abstract

Leveraging the power of multimodal large language models (LLMs) offers a promising approach to enhancing the accuracy and interpretability of morphing attack detection (MAD), especially in real-world biometric applications. This work introduces the use of LLMs for differential morphing attack detection (D-MAD). To the best of our knowledge, this is the first study to employ multimodal LLMs to D-MAD using real biometric data. To effectively utilize these models, we design Chain-of-Thought (CoT)-based prompts to reduce failure-to-answer rates and enhance the reasoning behind decisions. Our contributions include: (1) the first application of multimodal LLMs for D-MAD using real data subjects, (2) CoT-based prompt engineering to improve response reliability and explainability, (3) comprehensive qualitative and quantitative benchmarking of LLM performance using data from 54 individuals captured in passport enrollment scenarios, and (4) comparative analysis of two multimodal LLMs: ChatGPT-4o and Gemini providing insights into their morphing attack detection accuracy and decision transparency. Experimental results show that ChatGPT-4o outperforms Gemini in detection accuracy, especially against GAN-based morphs, though both models struggle under challenging conditions. While Gemini offers more consistent explanations, ChatGPT-4o is more resilient but prone to a higher failure-to-answer rate.