Diffusion Models for Imperceptible and Transferable Adversarial Attack

TL;DR

This paper introduces DiffAttack, a novel adversarial attack method using diffusion models to generate imperceptible, transferable perturbations in the latent space, outperforming existing techniques in various scenarios.

Contribution

It is the first to leverage diffusion models for adversarial attacks, enhancing imperceptibility and transferability by manipulating latent space and distracting model attention.

Findings

Outperforms existing attack methods in success rate

Generates human-insensitive, semantically meaningful perturbations

Effective against various models, datasets, and defenses

Abstract

Many existing adversarial attacks generate $L_p$-norm perturbations on image RGB space. Despite some achievements in transferability and attack success rate, the crafted adversarial examples are easily perceived by human eyes. Towards visual imperceptibility, some recent works explore unrestricted attacks without $L_p$-norm constraints, yet lacking transferability of attacking black-box models. In this work, we propose a novel imperceptible and transferable attack by leveraging both the generative and discriminative power of diffusion models. Specifically, instead of direct manipulation in pixel space, we craft perturbations in the latent space of diffusion models. Combined with well-designed content-preserving structures, we can generate human-insensitive perturbations embedded with semantic clues. For better transferability, we further "deceive" the diffusion model which can be viewed as an implicit recognition surrogate, by distracting its attention away from the target regions. To our knowledge, our proposed method, DiffAttack, is the first that introduces diffusion models into the adversarial attack field. Extensive experiments on various model structures, datasets, and defense methods have demonstrated the superiority of our attack over the existing attack methods.