Expanding the Role of Diffusion Models for Robust Classifier Training

TL;DR

This paper explores how diffusion models' internal representations can enhance adversarial training for robust image classifiers, showing that combining these representations with synthetic data improves robustness and feature disentanglement across multiple datasets.

Contribution

It introduces the novel idea of using diffusion model representations as auxiliary signals in adversarial training, beyond synthetic data generation.

Findings

Diffusion representations are diverse and partially robust.

Incorporating diffusion representations improves adversarial robustness.

Diffusion models encourage more disentangled features.

Abstract

Incorporating diffusion-generated synthetic data into adversarial training (AT) has been shown to substantially improve the training of robust image classifiers. In this work, we extend the role of diffusion models beyond merely generating synthetic data, examining whether their internal representations, which encode meaningful features of the data, can provide additional benefits for robust classifier training. Through systematic experiments, we show that diffusion models offer representations that are both diverse and partially robust, and that explicitly incorporating diffusion representations as an auxiliary learning signal during AT consistently improves robustness across settings. Furthermore, our representation analysis indicates that incorporating diffusion models into AT encourages more disentangled features, while diffusion representations and diffusion-generated synthetic data play complementary roles in shaping representations. Experiments on CIFAR-10, CIFAR-100, and ImageNet validate these findings, demonstrating the effectiveness of jointly leveraging diffusion representations and synthetic data within AT.