AlphaFace: High Fidelity and Real-time Face Swapper Robust to Facial Pose

TL;DR

AlphaFace is a real-time face swapping method that uses vision-language models and contrastive losses to improve identity preservation and attribute accuracy, especially in extreme facial poses.

Contribution

It introduces a novel approach combining vision-language models with contrastive learning to enhance face swapping robustness and real-time performance.

Findings

Outperforms state-of-the-art in pose-challenging scenarios

Maintains real-time processing speeds

Improves identity and attribute preservation

Abstract

Existing face-swapping methods often deliver competitive results in constrained settings but exhibit substantial quality degradation when handling extreme facial poses. To improve facial pose robustness, explicit geometric features are applied, but this approach remains problematic since it introduces additional dependencies and increases computational cost. Diffusion-based methods have achieved remarkable results; however, they are impractical for real-time processing. We introduce AlphaFace, which leverages an open-source vision-language model and CLIP image and text embeddings to apply novel visual and textual semantic contrastive losses. AlphaFace enables stronger identity representation and more precise attribute preservation, all while maintaining real-time performance. Comprehensive experiments across FF++, MPIE, and LPFF demonstrate that AlphaFace surpasses state-of-the-art methods in pose-challenging cases. The project is publicly available on `https://github.com/andrewyu90/Alphaface_Official.git'.