Synthetic to Authentic: Transferring Realism to 3D Face Renderings for Boosting Face Recognition

TL;DR

This paper explores using image-to-image translation to enhance the realism of 3D-rendered facial images, significantly improving face recognition performance trained on synthetic data when tested on real-world benchmarks.

Contribution

It introduces a method to transfer realism to synthetic 3D face images, boosting face recognition accuracy without relying solely on real datasets.

Findings

Realism transfer improves face recognition performance on real benchmarks.

Synthetic data with transferred realism outperforms non-realistic synthetic data.

The approach enables better use of synthetic data in practical face recognition applications.

Abstract

In this paper, we investigate the potential of image-to-image translation (I2I) techniques for transferring realism to 3D-rendered facial images in the context of Face Recognition (FR) systems. The primary motivation for using 3D-rendered facial images lies in their ability to circumvent the challenges associated with collecting large real face datasets for training FR systems. These images are generated entirely by 3D rendering engines, facilitating the generation of synthetic identities. However, it has been observed that FR systems trained on such synthetic datasets underperform when compared to those trained on real datasets, on various FR benchmarks. In this work, we demonstrate that by transferring the realism to 3D-rendered images (i.e., making the 3D-rendered images look more real), we can boost the performance of FR systems trained on these more photorealistic images. This improvement is evident when these systems are evaluated against FR benchmarks utilizing real-world data, thereby paving new pathways for employing synthetic data in real-world applications.