TCDiff: Triple Condition Diffusion Model with 3D Constraints for Stylizing Synthetic Faces

TL;DR

TCDiff is a novel diffusion model that enhances synthetic face generation by applying 3D and 2D constraints, improving identity consistency and intra-class variance for better face recognition performance.

Contribution

The paper introduces TCDiff, a triple condition diffusion model that incorporates 3D and 2D constraints to improve synthetic face style transfer and identity preservation.

Findings

Outperforms state-of-the-art synthetic datasets on face recognition benchmarks.

Enhances face identity consistency while maintaining intra-class variance.

Uses 3D and 2D facial constraints for improved style transfer.

Abstract

A robust face recognition model must be trained using datasets that include a large number of subjects and numerous samples per subject under varying conditions (such as pose, expression, age, noise, and occlusion). Due to ethical and privacy concerns, large-scale real face datasets have been discontinued, such as MS1MV3, and synthetic face generators have been proposed, utilizing GANs and Diffusion Models, such as SYNFace, SFace, DigiFace-1M, IDiff-Face, DCFace, and GANDiffFace, aiming to supply this demand. Some of these methods can produce high-fidelity realistic faces, but with low intra-class variance, while others generate high-variance faces with low identity consistency. In this paper, we propose a Triple Condition Diffusion Model (TCDiff) to improve face style transfer from real to synthetic faces through 2D and 3D facial constraints, enhancing face identity consistency while keeping the necessary high intra-class variance. Face recognition experiments using 1k, 2k, and 5k classes of our new dataset for training outperform state-of-the-art synthetic datasets in real face benchmarks such as LFW, CFP-FP, AgeDB, and BUPT. Our source code is available at: https://github.com/BOVIFOCR/tcdiff.