Pixel Is Not a Barrier: An Effective Evasion Attack for Pixel-Domain Diffusion Models

TL;DR

This paper introduces AtkPDM, a novel attack framework that effectively compromises pixel-domain diffusion models used in image editing, highlighting vulnerabilities and extending to latent diffusion models.

Contribution

The work presents the first effective attack method targeting pixel-domain diffusion models, exploiting denoising UNet vulnerabilities and improving naturalness of adversarial images.

Findings

AtkPDM successfully attacks PDM-based editing methods like SDEdit.

The framework maintains image fidelity and robustness against defenses.

It also extends to latent diffusion models with comparable performance.

Abstract

Diffusion Models have emerged as powerful generative models for high-quality image synthesis, with many subsequent image editing techniques based on them. However, the ease of text-based image editing introduces significant risks, such as malicious editing for scams or intellectual property infringement. Previous works have attempted to safeguard images from diffusion-based editing by adding imperceptible perturbations. These methods are costly and specifically target prevalent Latent Diffusion Models (LDMs), while Pixel-domain Diffusion Models (PDMs) remain largely unexplored and robust against such attacks. Our work addresses this gap by proposing a novel attack framework, AtkPDM. AtkPDM is mainly composed of a feature representation attacking loss that exploits vulnerabilities in denoising UNets and a latent optimization strategy to enhance the naturalness of adversarial images. Extensive experiments demonstrate the effectiveness of our approach in attacking dominant PDM-based editing methods (e.g., SDEdit) while maintaining reasonable fidelity and robustness against common defense methods. Additionally, our framework is extensible to LDMs, achieving comparable performance to existing approaches.