AdvLogo: Adversarial Patch Attack against Object Detectors based on Diffusion Models

TL;DR

AdvLogo introduces a novel semantic-based adversarial patch attack on object detectors using diffusion models, effectively balancing attack success with high visual quality by perturbing latent embeddings in the frequency domain.

Contribution

It proposes a new semantic perspective for adversarial patch attacks leveraging diffusion models and frequency domain perturbations to improve attack effectiveness and visual quality.

Findings

Achieves strong attack performance against object detectors.

Maintains high visual quality of adversarial patches.

Utilizes diffusion model's semantic understanding for attack generation.

Abstract

With the rapid development of deep learning, object detectors have demonstrated impressive performance; however, vulnerabilities still exist in certain scenarios. Current research exploring the vulnerabilities using adversarial patches often struggles to balance the trade-off between attack effectiveness and visual quality. To address this problem, we propose a novel framework of patch attack from semantic perspective, which we refer to as AdvLogo. Based on the hypothesis that every semantic space contains an adversarial subspace where images can cause detectors to fail in recognizing objects, we leverage the semantic understanding of the diffusion denoising process and drive the process to adversarial subareas by perturbing the latent and unconditional embeddings at the last timestep. To mitigate the distribution shift that exposes a negative impact on image quality, we apply perturbation to the latent in frequency domain with the Fourier Transform. Experimental results demonstrate that AdvLogo achieves strong attack performance while maintaining high visual quality.