Efficient Image-to-Image Diffusion Classifier for Adversarial Robustness

TL;DR

This paper introduces an efficient diffusion-based image classifier that enhances adversarial robustness while reducing computational costs by redesigning the diffusion process for classification tasks.

Contribution

The authors propose a novel diffusion framework for image classification that reduces complexity and computational costs, improving adversarial robustness compared to existing diffusion and CNN methods.

Findings

Achieves better adversarial robustness with fewer computations

Reduces diffusion timesteps and network size without sacrificing accuracy

Outperforms traditional CNN-based classifiers under various attacks

Abstract

Diffusion models (DMs) have demonstrated great potential in the field of adversarial robustness, where DM-based defense methods can achieve superior defense capability without adversarial training. However, they all require huge computational costs due to the usage of large-scale pre-trained DMs, making it difficult to conduct full evaluation under strong attacks and compare with traditional CNN-based methods. Simply reducing the network size and timesteps in DMs could significantly harm the image generation quality, which invalidates previous frameworks. To alleviate this issue, we redesign the diffusion framework from generating high-quality images to predicting distinguishable image labels. Specifically, we employ an image translation framework to learn many-to-one mapping from input samples to designed orthogonal image labels. Based on this framework, we introduce an efficient Image-to-Image diffusion classifier with a pruned U-Net structure and reduced diffusion timesteps. Besides the framework, we redesign the optimization objective of DMs to fit the target of image classification, where a new classification loss is incorporated in the DM-based image translation framework to distinguish the generated label from those of other classes. We conduct sufficient evaluations of the proposed classifier under various attacks on popular benchmarks. Extensive experiments show that our method achieves better adversarial robustness with fewer computational costs than DM-based and CNN-based methods. The code is available at https://github.com/hfmei/IDC