Multimodal Biometric Authentication Using Camera-Based PPG and Fingerprint Fusion

TL;DR

This paper introduces a multimodal biometric authentication system combining camera-based PPG signals and fingerprint data, utilizing neural networks with attention mechanisms to improve verification accuracy.

Contribution

It presents a novel fusion approach of PPG and fingerprint biometrics using neural networks with cross-modal attention and contrastive loss for enhanced security.

Findings

Superior authentication accuracy demonstrated in experiments

Effective fusion of PPG and fingerprint modalities

Robust performance in single and dual-session scenarios

Abstract

Camera-based photoplethysmography (PPG) obtained from smartphones has shown great promise for personalized healthcare and secure authentication. This paper presents a multimodal biometric system that integrates PPG signals extracted from videos with fingerprint data to enhance the accuracy of user verification. The system requires users to place their fingertip on the camera lens for a few seconds, allowing the capture and processing of unique biometric characteristics. Our approach employs a neural network with two structured state-space model (SSM) encoders to manage the distinct modalities. Fingerprint images are transformed into pixel sequences, and along with segmented PPG waveforms, they are input into the encoders. A cross-modal attention mechanism then extracts refined feature representations, and a distribution-oriented contrastive loss function aligns these features within a unified latent space. Experimental results demonstrate the system's superior performance across various evaluation metrics in both single-session and dual-session authentication scenarios.