BPFNet: A Unified Framework for Bimodal Palmprint Alignment and Fusion

TL;DR

BPFNet is an end-to-end framework that improves bimodal palmprint recognition by directly localizing ROIs, aligning images, and fusing palmprint and palm vein data, achieving state-of-the-art results.

Contribution

The paper introduces BPFNet, a novel unified framework for ROI detection, alignment, and fusion in bimodal palmprint recognition, addressing limitations of keypoint-based methods.

Findings

Achieves state-of-the-art accuracy on CUHKSZ-v1 and TongJi datasets.

Effectively localizes ROIs and aligns palmprint images without keypoint detection.

Demonstrates superior bimodal fusion performance through a novel cross-modal scheme.

Abstract

Bimodal palmprint recognition leverages palmprint and palm vein images simultaneously,which achieves high accuracy by multi-model information fusion and has strong anti-falsification property. In the recognition pipeline, the detection of palm and the alignment of region-of-interest (ROI) are two crucial steps for accurate matching. Most existing methods localize palm ROI by keypoint detection algorithms, however the intrinsic difficulties of keypoint detection tasks make the results unsatisfactory. Besides, the ROI alignment and fusion algorithms at image-level are not fully investigaged.To bridge the gap, in this paper, we propose Bimodal Palmprint Fusion Network (BPFNet) which focuses on ROI localization, alignment and bimodal image fusion.BPFNet is an end-to-end framework containing two subnets: The detection network directly regresses the palmprint ROIs based on bounding box prediction and conducts alignment by translation estimation.In the downstream,the bimodal fusion network implements bimodal ROI image fusion leveraging a novel proposed cross-modal selection scheme. To show the effectiveness of BPFNet,we carry out experiments on the large-scale touchless palmprint datasets CUHKSZ-v1 and TongJi and the proposed method achieves state-of-the-art performances.