Diffusion to Confusion: Naturalistic Adversarial Patch Generation Based on Diffusion Model for Object Detector

TL;DR

This paper introduces a novel diffusion model-based method for generating naturalistic adversarial patches that effectively deceive object detectors while maintaining high visual realism, outperforming existing techniques.

Contribution

First to utilize diffusion models for creating naturalistic adversarial patches targeting object detectors, improving stealthiness and attack stability over prior methods.

Findings

Generated patches are more natural and visually convincing.

Achieved better attack success rates compared to state-of-the-art methods.

Demonstrated stable and high-quality patch generation across various conditions.

Abstract

Many physical adversarial patch generation methods are widely proposed to protect personal privacy from malicious monitoring using object detectors. However, they usually fail to generate satisfactory patch images in terms of both stealthiness and attack performance without making huge efforts on careful hyperparameter tuning. To address this issue, we propose a novel naturalistic adversarial patch generation method based on the diffusion models (DM). Through sampling the optimal image from the DM model pretrained upon natural images, it allows us to stably craft high-quality and naturalistic physical adversarial patches to humans without suffering from serious mode collapse problems as other deep generative models. To the best of our knowledge, we are the first to propose DM-based naturalistic adversarial patch generation for object detectors. With extensive quantitative, qualitative, and subjective experiments, the results demonstrate the effectiveness of the proposed approach to generate better-quality and more naturalistic adversarial patches while achieving acceptable attack performance than other state-of-the-art patch generation methods. We also show various generation trade-offs under different conditions.