HEFT: Homomorphically Encrypted Fusion of Biometric Templates

TL;DR

HEFT introduces a fully homomorphic encryption-based method for secure biometric template fusion and matching, enabling privacy-preserving biometric verification with improved accuracy and efficiency.

Contribution

The paper presents HEFT, a novel FHE-compatible biometric fusion and matching framework with custom operations, efficient data packing, and an FHE-aware training algorithm.

Findings

Improves biometric verification AUROC by over 9-11%.

Compresses feature vectors by a factor of 16.

Performs encrypted match scoring in under 900 ms.

Abstract

This paper proposes a non-interactive end-to-end solution for secure fusion and matching of biometric templates using fully homomorphic encryption (FHE). Given a pair of encrypted feature vectors, we perform the following ciphertext operations, i) feature concatenation, ii) fusion and dimensionality reduction through a learned linear projection, iii) scale normalization to unit $\ell_2$-norm, and iv) match score computation. Our method, dubbed HEFT (Homomorphically Encrypted Fusion of biometric Templates), is custom-designed to overcome the unique constraint imposed by FHE, namely the lack of support for non-arithmetic operations. From an inference perspective, we systematically explore different data packing schemes for computationally efficient linear projection and introduce a polynomial approximation for scale normalization. From a training perspective, we introduce an FHE-aware algorithm for learning the linear projection matrix to mitigate errors induced by approximate normalization. Experimental evaluation for template fusion and matching of face and voice biometrics shows that HEFT (i) improves biometric verification performance by 11.07% and 9.58% AUROC compared to the respective unibiometric representations while compressing the feature vectors by a factor of 16 (512D to 32D), and (ii) fuses a pair of encrypted feature vectors and computes its match score against a gallery of size 1024 in 884 ms. Code and data are available at https://github.com/human-analysis/encrypted-biometric-fusion