My Face Is Mine, Not Yours: Facial Protection Against Diffusion Model Face Swapping

TL;DR

This paper proposes a proactive adversarial defense method to protect facial images from being exploited by diffusion-based deepfake models, addressing the limitations of existing approaches that target traditional generative architectures.

Contribution

It introduces a diffusion-specific adversarial protection technique that is robust across various diffusion models and focuses on region-specific perturbations for facial protection.

Findings

Effective in preventing diffusion-based face swapping

Robust across multiple diffusion model architectures

Outperforms existing passive detection methods

Abstract

The proliferation of diffusion-based deepfake technologies poses significant risks for unauthorized and unethical facial image manipulation. While traditional countermeasures have primarily focused on passive detection methods, this paper introduces a novel proactive defense strategy through adversarial attacks that preemptively protect facial images from being exploited by diffusion-based deepfake systems. Existing adversarial protection methods predominantly target conventional generative architectures (GANs, AEs, VAEs) and fail to address the unique challenges presented by diffusion models, which have become the predominant framework for high-quality facial deepfakes. Current diffusion-specific adversarial approaches are limited by their reliance on specific model architectures and weights, rendering them ineffective against the diverse landscape of diffusion-based deepfake implementations. Additionally, they typically employ global perturbation strategies that inadequately address the region-specific nature of facial manipulation in deepfakes.