Generalist Multimodal LLMs Gain Biometric Expertise via Human Salience

TL;DR

This paper demonstrates that general-purpose multimodal large language models, when augmented with human expert knowledge, can effectively perform iris presentation attack detection within privacy constraints, outperforming specialized models and human examiners.

Contribution

It shows that pre-trained vision transformers in MLLMs can cluster iris attack types and, with structured prompts, resolve ambiguities, enabling effective iris PAD without sharing biometric data.

Findings

Gemini with expert prompts outperforms CNN baseline and human examiners.

Locally-deployable Llama achieves near-human performance.

Pre-trained vision transformers inherently cluster attack types.

Abstract

Iris presentation attack detection (PAD) is critical for secure biometric deployments, yet developing specialized models faces significant practical barriers: collecting data representing future unknown attacks is impossible, and collecting diverse-enough data, yet still limited in terms of its predictive power, is expensive. Additionally, sharing biometric data raises privacy concerns. Due to rapid emergence of new attack vectors demanding adaptable solutions, we thus investigate in this paper whether general-purpose multimodal large language models (MLLMs) can perform iris PAD when augmented with human expert knowledge, operating under strict privacy constraints that prohibit sending biometric data to public cloud MLLM services. Through analysis of vision encoder embeddings applied to our dataset, we demonstrate that pre-trained vision transformers in MLLMs inherently cluster many iris attack types despite never being explicitly trained for this task. However, where clustering shows overlap between attack classes, we find that structured prompts incorporating human salience (verbal descriptions from subjects identifying attack indicators) enable these models to resolve ambiguities. Testing on an IRB-restricted dataset of 224 iris images spanning seven attack types, using only university-approved services (Gemini 2.5 Pro) or locally-hosted models (e.g., Llama 3.2-Vision), we show that Gemini with expert-informed prompts outperforms both a specialized convolutional neural networks (CNN)-based baseline and human examiners, while the locally-deployable Llama achieves near-human performance. Our results establish that MLLMs deployable within institutional privacy constraints offer a viable path for iris PAD.