VFace: A Training-Free Approach for Diffusion-Based Video Face Swapping

TL;DR

VFace is a training-free, modular approach for high-quality, temporally consistent face swapping in videos using diffusion models, employing novel attention techniques without additional training.

Contribution

It introduces a training-free, plug-and-play framework with novel attention mechanisms for improved video face swapping quality and temporal coherence.

Findings

Significantly improves temporal consistency in video face swapping.

Enhances visual fidelity without additional training.

Operates seamlessly with existing diffusion-based methods.

Abstract

We present a training-free, plug-and-play method, namely VFace, for high-quality face swapping in videos. It can be seamlessly integrated with image-based face swapping approaches built on diffusion models. First, we introduce a Frequency Spectrum Attention Interpolation technique to facilitate generation and intact key identity characteristics. Second, we achieve Target Structure Guidance via plug-and-play attention injection to better align the structural features from the target frame to the generation. Third, we present a Flow-Guided Attention Temporal Smoothening mechanism that enforces spatiotemporal coherence without modifying the underlying diffusion model to reduce temporal inconsistencies typically encountered in frame-wise generation. Our method requires no additional training or video-specific fine-tuning. Extensive experiments show that our method significantly enhances temporal consistency and visual fidelity, offering a practical and modular solution for video-based face swapping. Our code is available at https://github.com/Sanoojan/VFace.