CanonSwap: High-Fidelity and Consistent Video Face Swapping via Canonical Space Modulation

TL;DR

CanonSwap introduces a novel video face swapping framework that decouples motion and appearance information in a canonical space, enabling high-fidelity, consistent, and realistic identity transfer while preserving dynamic facial attributes.

Contribution

The paper proposes CanonSwap, a new method that separates motion from appearance for improved identity transfer and dynamic attribute preservation in video face swapping.

Findings

Outperforms existing methods in visual quality and temporal consistency

Achieves superior identity preservation with minimal artifacts

Provides comprehensive metrics for evaluating face swapping performance

Abstract

Video face swapping aims to address two primary challenges: effectively transferring the source identity to the target video and accurately preserving the dynamic attributes of the target face, such as head poses, facial expressions, lip-sync, \etc. Existing methods mainly focus on achieving high-quality identity transfer but often fall short in maintaining the dynamic attributes of the target face, leading to inconsistent results. We attribute this issue to the inherent coupling of facial appearance and motion in videos. To address this, we propose CanonSwap, a novel video face-swapping framework that decouples motion information from appearance information. Specifically, CanonSwap first eliminates motion-related information, enabling identity modification within a unified canonical space. Subsequently, the swapped feature is reintegrated into the original video space, ensuring the preservation of the target face's dynamic attributes. To further achieve precise identity transfer with minimal artifacts and enhanced realism, we design a Partial Identity Modulation module that adaptively integrates source identity features using a spatial mask to restrict modifications to facial regions. Additionally, we introduce several fine-grained synchronization metrics to comprehensively evaluate the performance of video face swapping methods. Extensive experiments demonstrate that our method significantly outperforms existing approaches in terms of visual quality, temporal consistency, and identity preservation. Our project page are publicly available at https://luoxyhappy.github.io/CanonSwap/.