Realistic Face Reconstruction from Facial Embeddings via Diffusion Models

TL;DR

This paper introduces a novel framework using diffusion models to reconstruct realistic high-resolution face images from facial embeddings, revealing privacy risks in face recognition systems.

Contribution

The work proposes the face embedding mapping (FEM) framework utilizing Kolmogorov-Arnold Network and diffusion models for embedding-to-face reconstruction, highlighting privacy vulnerabilities.

Findings

Reconstructed faces can access other face recognition systems.

The method is robust against partial and protected embeddings.

FEM can evaluate privacy leakage in face recognition systems.

Abstract

With the advancement of face recognition (FR) systems, privacy-preserving face recognition (PPFR) systems have gained popularity for their accurate recognition, enhanced facial privacy protection, and robustness to various attacks. However, there are limited studies to further verify privacy risks by reconstructing realistic high-resolution face images from embeddings of these systems, especially for PPFR. In this work, we propose the face embedding mapping (FEM), a general framework that explores Kolmogorov-Arnold Network (KAN) for conducting the embedding-to-face attack by leveraging pre-trained Identity-Preserving diffusion model against state-of-the-art (SOTA) FR and PPFR systems. Based on extensive experiments, we verify that reconstructed faces can be used for accessing other real-word FR systems. Besides, the proposed method shows the robustness in reconstructing faces from the partial and protected face embeddings. Moreover, FEM can be utilized as a tool for evaluating safety of FR and PPFR systems in terms of privacy leakage. All images used in this work are from public datasets.