Gradient-Free Adversarial Purification with Diffusion Models

TL;DR

This paper presents a novel, efficient defense framework against both perturbation-based and unrestricted adversarial attacks, combining adversarial anti-aliasing, super-resolution, and contrastive learning-based fine-tuning without retraining diffusion models.

Contribution

It introduces a gradient-free adversarial purification method using diffusion models, combining preprocessing and fine-tuning techniques for robust defense.

Findings

Effective against diverse adversarial attacks

No additional training required for purification techniques

Enhanced robustness through dataset-specific fine-tuning

Abstract

Adversarial training and adversarial purification are two widely used defense strategies for enhancing model robustness against adversarial attacks. However, adversarial training requires costly retraining, while adversarial purification often suffers from low efficiency. More critically, existing defenses are primarily designed under the perturbation-based adversarial threat model, which is ineffective against recently introduced unrestricted adversarial attacks. In this paper, we propose an effective and efficient defense framework that counters both perturbation-based and unrestricted adversarial attacks. Our approach is motivated by the observation that adversarial examples typically lie near the decision boundary and are highly sensitive to pixel-level perturbations. To address this, we introduce adversarial anti-aliasing, a preprocessing technique that mitigates adversarial noise by reducing the magnitude of pixel-level perturbations. In addition, we propose adversarial super-resolution, which leverages prior knowledge from clean datasets to benignly restore high-quality images from adversarially degraded ones. Unlike image synthesis methods that generate entirely new images, adversarial super-resolution focuses on image restoration, making it more suitable for purification. Importantly, both techniques require no additional training and are computationally efficient since they do not rely on gradient computations. To further improve robustness across diverse datasets, we introduce a contrastive learning-based adversarial deblurring fine-tuning method. By incorporating adversarial priors during fine-tuning on the target dataset, this method enhances purification effectiveness without the need to retrain diffusion models.