My Face Is Mine, Not Yours: Facial Protection Against Diffusion Model Face Swapping
Hon Ming Yam, Zhongliang Guo, Chun Pong Lau

TL;DR
This paper proposes a proactive adversarial defense method to protect facial images from being exploited by diffusion-based deepfake models, addressing the limitations of existing approaches that target traditional generative architectures.
Contribution
It introduces a diffusion-specific adversarial protection technique that is robust across various diffusion models and focuses on region-specific perturbations for facial protection.
Findings
Effective in preventing diffusion-based face swapping
Robust across multiple diffusion model architectures
Outperforms existing passive detection methods
Abstract
The proliferation of diffusion-based deepfake technologies poses significant risks for unauthorized and unethical facial image manipulation. While traditional countermeasures have primarily focused on passive detection methods, this paper introduces a novel proactive defense strategy through adversarial attacks that preemptively protect facial images from being exploited by diffusion-based deepfake systems. Existing adversarial protection methods predominantly target conventional generative architectures (GANs, AEs, VAEs) and fail to address the unique challenges presented by diffusion models, which have become the predominant framework for high-quality facial deepfakes. Current diffusion-specific adversarial approaches are limited by their reliance on specific model architectures and weights, rendering them ineffective against the diverse landscape of diffusion-based deepfake…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFace recognition and analysis · Facial Nerve Paralysis Treatment and Research · Biometric Identification and Security
MethodsDiffusion
