Exploring Adversarial Attacks against Latent Diffusion Model from the Perspective of Adversarial Transferability

TL;DR

This paper investigates how the properties of surrogate models, particularly their smoothness, affect the transferability and effectiveness of adversarial examples against latent diffusion models, supported by theoretical analysis.

Contribution

It introduces the perspective of surrogate model smoothness in adversarial transferability for latent diffusion models and demonstrates improved attack performance through smoother surrogate model selection.

Findings

Smoother surrogate models enhance adversarial transferability.

Selecting smoother models significantly improves attack success rates.

Theoretical analysis explains the role of model smoothness in transferability.

Abstract

Recently, many studies utilized adversarial examples (AEs) to raise the cost of malicious image editing and copyright violation powered by latent diffusion models (LDMs). Despite their successes, a few have studied the surrogate model they used to generate AEs. In this paper, from the perspective of adversarial transferability, we investigate how the surrogate model's property influences the performance of AEs for LDMs. Specifically, we view the time-step sampling in the Monte-Carlo-based (MC-based) adversarial attack as selecting surrogate models. We find that the smoothness of surrogate models at different time steps differs, and we substantially improve the performance of the MC-based AEs by selecting smoother surrogate models. In the light of the theoretical framework on adversarial transferability in image classification, we also conduct a theoretical analysis to explain why smooth surrogate models can also boost AEs for LDMs.