Shielding Latent Face Representations From Privacy Attacks

TL;DR

This paper proposes a multi-layer privacy protection framework for face embeddings that combines Fully Homomorphic Encryption and hashing, effectively safeguarding sensitive attributes while preserving face recognition capabilities.

Contribution

It introduces a novel multi-layer protection method integrating FHE and hashing to secure face embeddings against privacy attacks.

Findings

Outperforms state-of-the-art privacy protection methods

Reduces encrypted processing overhead through embedding compression

Maintains face identification accuracy while protecting sensitive attributes

Abstract

In today's data-driven analytics landscape, deep learning has become a powerful tool, with latent representations, known as embeddings, playing a central role in several applications. In the face analytics domain, such embeddings are commonly used for biometric recognition (e.g., face identification). However, these embeddings, or templates, can inadvertently expose sensitive attributes such as age, gender, and ethnicity. Leaking such information can compromise personal privacy and affect civil liberty and human rights. To address these concerns, we introduce a multi-layer protection framework for embeddings. It consists of a sequence of operations: (a) encrypting embeddings using Fully Homomorphic Encryption (FHE), and (b) hashing them using irreversible feature manifold hashing. Unlike conventional encryption methods, FHE enables computations directly on encrypted data, allowing downstream analytics while maintaining strong privacy guarantees. To reduce the overhead of encrypted processing, we employ embedding compression. Our proposed method shields latent representations of sensitive data from leaking private attributes (such as age and gender) while retaining essential functional capabilities (such as face identification). Extensive experiments on two datasets using two face encoders demonstrate that our approach outperforms several state-of-the-art privacy protection methods.