Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model

TL;DR

This paper introduces a semi-supervised adversarial framework that generates diverse, photorealistic face images of new identities with controlled pose, lighting, and expression, improving face recognition performance.

Contribution

It presents a novel semi-supervised adversarial approach with pairwise supervision for generating realistic face images conditioned on a 3D morphable model, enhancing diversity and identity preservation.

Findings

Generated images show high pose, lighting, and expression diversity.

Combining generated images with real data improves face recognition accuracy.

Achieves comparable results to state-of-the-art on LFW and IJB-A datasets.

Abstract

We propose a novel end-to-end semi-supervised adversarial framework to generate photorealistic face images of new identities with wide ranges of expressions, poses, and illuminations conditioned by a 3D morphable model. Previous adversarial style-transfer methods either supervise their networks with large volume of paired data or use unpaired data with a highly under-constrained two-way generative framework in an unsupervised fashion. We introduce pairwise adversarial supervision to constrain two-way domain adaptation by a small number of paired real and synthetic images for training along with the large volume of unpaired data. Extensive qualitative and quantitative experiments are performed to validate our idea. Generated face images of new identities contain pose, lighting and expression diversity and qualitative results show that they are highly constraint by the synthetic input image while adding photorealism and retaining identity information. We combine face images generated by the proposed method with the real data set to train face recognition algorithms. We evaluated the model on two challenging data sets: LFW and IJB-A. We observe that the generated images from our framework consistently improves over the performance of deep face recognition network trained with Oxford VGG Face dataset and achieves comparable results to the state-of-the-art.