Training face verification models from generated face identity data

TL;DR

This paper proposes a method to generate synthetic face identity data using a modified StyleGAN, aiming to enhance privacy in face recognition training while analyzing the trade-offs in model performance and privacy protection.

Contribution

It introduces a novel approach to create synthetic face data with controlled identity features to improve privacy in face recognition models.

Findings

Synthetic data provides good privacy protection against membership attacks.

Model performance degrades compared to state-of-the-art face verification.

Adding private data improves model accuracy significantly.

Abstract

Machine learning tools are becoming increasingly powerful and widely used. Unfortunately membership attacks, which seek to uncover information from data sets used in machine learning, have the potential to limit data sharing. In this paper we consider an approach to increase the privacy protection of data sets, as applied to face recognition. Using an auxiliary face recognition model, we build on the StyleGAN generative adversarial network and feed it with latent codes combining two distinct sub-codes, one encoding visual identity factors, and, the other, non-identity factors. By independently varying these vectors during image generation, we create a synthetic data set of fictitious face identities. We use this data set to train a face recognition model. The model performance degrades in comparison to the state-of-the-art of face verification. When tested with a simple membership attack our model provides good privacy protection, however the model performance degrades in comparison to the state-of-the-art of face verification. We find that the addition of a small amount of private data greatly improves the performance of our model, which highlights the limitations of using synthetic data to train machine learning models.