Improving Adversarial Robustness by Contrastive Guided Diffusion Process

TL;DR

This paper introduces Contrastive-Guided Diffusion Process, a method that uses contrastive loss to improve the quality of synthetic data generated by diffusion models, thereby enhancing adversarial robustness efficiently.

Contribution

It proposes a novel contrastive-guided diffusion approach that improves synthetic data distinguishability and robustness, addressing sample efficiency issues in adversarial training.

Findings

Contrastive-DP enhances synthetic data quality.

Improves adversarial robustness in image classification.

Theoretical analysis confirms the importance of data distinguishability.

Abstract

Synthetic data generation has become an emerging tool to help improve the adversarial robustness in classification tasks since robust learning requires a significantly larger amount of training samples compared with standard classification tasks. Among various deep generative models, the diffusion model has been shown to produce high-quality synthetic images and has achieved good performance in improving the adversarial robustness. However, diffusion-type methods are typically slow in data generation as compared with other generative models. Although different acceleration techniques have been proposed recently, it is also of great importance to study how to improve the sample efficiency of generated data for the downstream task. In this paper, we first analyze the optimality condition of synthetic distribution for achieving non-trivial robust accuracy. We show that enhancing the distinguishability among the generated data is critical for improving adversarial robustness. Thus, we propose the Contrastive-Guided Diffusion Process (Contrastive-DP), which adopts the contrastive loss to guide the diffusion model in data generation. We verify our theoretical results using simulations and demonstrate the good performance of Contrastive-DP on image datasets.