Heterogeneous Face Recognition via Face Synthesis with Identity-Attribute Disentanglement

TL;DR

This paper introduces a novel face synthesis approach with identity-attribute disentanglement to enhance heterogeneous face recognition, significantly improving cross-domain matching accuracy by augmenting data with synthetic images.

Contribution

The proposed FSIAD method uniquely combines identity-attribute disentanglement with face synthesis to generate diverse training data, advancing HFR performance beyond existing methods.

Findings

Achieves 4.8% improvement in VR@FAR=0.01% on LAMP-HQ

Outperforms previous HFR approaches on five databases

Enriches training data with synthetic images for better generalization

Abstract

Heterogeneous Face Recognition (HFR) aims to match faces across different domains (e.g., visible to near-infrared images), which has been widely applied in authentication and forensics scenarios. However, HFR is a challenging problem because of the large cross-domain discrepancy, limited heterogeneous data pairs, and large variation of facial attributes. To address these challenges, we propose a new HFR method from the perspective of heterogeneous data augmentation, named Face Synthesis with Identity-Attribute Disentanglement (FSIAD). Firstly, the identity-attribute disentanglement (IAD) decouples face images into identity-related representations and identity-unrelated representations (called attributes), and then decreases the correlation between identities and attributes. Secondly, we devise a face synthesis module (FSM) to generate a large number of images with stochastic combinations of disentangled identities and attributes for enriching the attribute diversity of synthetic images. Both the original images and the synthetic ones are utilized to train the HFR network for tackling the challenges and improving the performance of HFR. Extensive experiments on five HFR databases validate that FSIAD obtains superior performance than previous HFR approaches. Particularly, FSIAD obtains 4.8% improvement over state of the art in terms of VR@FAR=0.01% on LAMP-HQ, the largest HFR database so far.