Generative Robust Classification

TL;DR

This paper explores using adversarial training to develop generative classifiers that can improve robustness against larger test perturbations and better model class-conditional distributions, offering a promising alternative to traditional discriminative methods.

Contribution

It introduces a generative adversarial training approach for robust classification that matches discriminative models on clean data and outperforms them on larger perturbations.

Findings

Generative classifiers perform comparably on clean data and better on large perturbations.

They can generate realistic samples and counterfactuals.

Advanced data augmentation improves robustness in the proposed method.

Abstract

Training adversarially robust discriminative (i.e., softmax) classifier has been the dominant approach to robust classification. Building on recent work on adversarial training (AT)-based generative models, we investigate using AT to learn unnormalized class-conditional density models and then performing generative robust classification. Our result shows that, under the condition of similar model capacities, the generative robust classifier achieves comparable performance to a baseline softmax robust classifier when the test data is clean or when the test perturbation is of limited size, and much better performance when the test perturbation size exceeds the training perturbation size. The generative classifier is also able to generate samples or counterfactuals that more closely resemble the training data, suggesting that the generative classifier can better capture the class-conditional distributions. In contrast to standard discriminative adversarial training where advanced data augmentation techniques are only effective when combined with weight averaging, we find it straightforward to apply advanced data augmentation to achieve better robustness in our approach. Our result suggests that the generative classifier is a competitive alternative to robust classification, especially for problems with limited number of classes.