MotionSwap

TL;DR

This paper improves the SimSwap face swapping framework by integrating attention mechanisms, dynamic loss weighting, and learning rate scheduling, resulting in higher fidelity, identity preservation, and visual quality.

Contribution

The paper introduces architectural and training enhancements to SimSwap, achieving significant performance improvements over the baseline.

Findings

Better identity similarity and attribute consistency

Lower FID scores and improved visual quality

Ablation studies confirming the effectiveness of each enhancement

Abstract

Face swapping technology has gained significant attention in both academic research and commercial applications. This paper presents our implementation and enhancement of SimSwap, an efficient framework for high fidelity face swapping. We introduce several improvements to the original model, including the integration of self and cross-attention mechanisms in the generator architecture, dynamic loss weighting, and cosine annealing learning rate scheduling. These enhancements lead to significant improvements in identity preservation, attribute consistency, and overall visual quality. Our experimental results, spanning 400,000 training iterations, demonstrate progressive improvements in generator and discriminator performance. The enhanced model achieves better identity similarity, lower FID scores, and visibly superior qualitative results compared to the baseline. Ablation studies confirm the importance of each architectural and training improvement. We conclude by identifying key future directions, such as integrating StyleGAN3, improving lip synchronization, incorporating 3D facial modeling, and introducing temporal consistency for video-based applications.