Finger Pose Estimation for Under-screen Fingerprint Sensor

TL;DR

This paper introduces a dual-modal neural network for under-screen fingerprint pose estimation, combining texture and contour data to improve accuracy and robustness in challenging conditions.

Contribution

It proposes a novel dual-modal input network with a decoupled probability prediction, MoE feature fusion, and cross-domain knowledge transfer for enhanced fingerprint pose estimation.

Findings

Outperforms previous SOTA methods on public and private datasets.

Significantly improves fingerprint recognition accuracy.

Demonstrates robustness to large angles and small areas.

Abstract

Two-dimensional pose estimation plays a crucial role in fingerprint recognition by facilitating global alignment and reduce pose-induced variations. However, existing methods are still unsatisfactory when handling with large angle or small area inputs. These limitations are particularly pronounced on fingerprints captured by under-screen fingerprint sensors in smartphones. In this paper, we present a novel dual-modal input based network for under-screen fingerprint pose estimation. Our approach effectively integrates two distinct yet complementary modalities: texture details extracted from ridge patches through the under-screen fingerprint sensor, and rough contours derived from capacitive images obtained via the touch screen. This collaborative integration endows our network with more comprehensive and discriminative information, substantially improving the accuracy and stability of pose estimation. A decoupled probability distribution prediction task is designed, instead of the traditional supervised forms of numerical regression or heatmap voting, to facilitate the training process. Additionally, we incorporate a Mixture of Experts (MoE) based feature fusion mechanism and a relationship driven cross-domain knowledge transfer strategy to further strengthen feature extraction and fusion capabilities. Extensive experiments are conducted on several public datasets and two private datasets. The results indicate that our method is significantly superior to previous state-of-the-art (SOTA) methods and remarkably boosts the recognition ability of fingerprint recognition algorithms. Our code is available at https://github.com/XiongjunGuan/DRACO.