DiffSwap++: 3D Latent-Controlled Diffusion for Identity-Preserving Face Swapping

TL;DR

DiffSwap++ introduces a 3D-aware diffusion-based face swapping method that significantly improves identity preservation and geometric consistency by leveraging 3D facial features during training and generation.

Contribution

It is the first to incorporate 3D facial latent features into a diffusion-based face swapping pipeline, enhancing disentanglement and realism.

Findings

Outperforms prior methods in identity preservation.

Maintains target pose and expression effectively.

Validated by biometric evaluation and user study.

Abstract

Diffusion-based approaches have recently achieved strong results in face swapping, offering improved visual quality over traditional GAN-based methods. However, even state-of-the-art models often suffer from fine-grained artifacts and poor identity preservation, particularly under challenging poses and expressions. A key limitation of existing approaches is their failure to meaningfully leverage 3D facial structure, which is crucial for disentangling identity from pose and expression. In this work, we propose DiffSwap++, a novel diffusion-based face-swapping pipeline that incorporates 3D facial latent features during training. By guiding the generation process with 3D-aware representations, our method enhances geometric consistency and improves the disentanglement of facial identity from appearance attributes. We further design a diffusion architecture that conditions the denoising process on both identity embeddings and facial landmarks, enabling high-fidelity and identity-preserving face swaps. Extensive experiments on CelebA, FFHQ, and CelebV-Text demonstrate that DiffSwap++ outperforms prior methods in preserving source identity while maintaining target pose and expression. Additionally, we introduce a biometric-style evaluation and conduct a user study to further validate the realism and effectiveness of our approach. Code will be made publicly available at https://github.com/WestonBond/DiffSwapPP