Controllable and Guided Face Synthesis for Unconstrained Face Recognition

TL;DR

This paper introduces a controllable face synthesis model that generates diverse, dataset-like face images to improve unconstrained face recognition performance, guided by a face recognition model.

Contribution

It proposes a novel controllable face synthesis method that mimics target dataset distributions and enhances face recognition training in unconstrained environments.

Findings

Achieves over 5.76% Rank1 improvement on benchmarks.

Provides a way to measure dataset similarity via learned bases.

Enables precise control over synthesis diversity.

Abstract

Although significant advances have been made in face recognition (FR), FR in unconstrained environments remains challenging due to the domain gap between the semi-constrained training datasets and unconstrained testing scenarios. To address this problem, we propose a controllable face synthesis model (CFSM) that can mimic the distribution of target datasets in a style latent space. CFSM learns a linear subspace with orthogonal bases in the style latent space with precise control over the diversity and degree of synthesis. Furthermore, the pre-trained synthesis model can be guided by the FR model, making the resulting images more beneficial for FR model training. Besides, target dataset distributions are characterized by the learned orthogonal bases, which can be utilized to measure the distributional similarity among face datasets. Our approach yields significant performance gains on unconstrained benchmarks, such as IJB-B, IJB-C, TinyFace and IJB-S (+5.76% Rank1).