Deep Repulsive Prototypes for Adversarial Robustness

TL;DR

This paper introduces deep repulsive prototypes, a novel training method that enhances adversarial robustness by increasing class separation in output space, achieving competitive results without the high costs of adversarial training.

Contribution

The paper proposes a new approach to improve adversarial robustness through output space partitioning into prototypes, eliminating the need for adversarial training.

Findings

Achieved over 50% robustness on CIFAR-10 with 92% natural accuracy.

Obtained over 20% robustness on CIFAR-100 with 71% natural accuracy.

Models showed increased resilience to large perturbations compared to adversarial training.

Abstract

While many defences against adversarial examples have been proposed, finding robust machine learning models is still an open problem. The most compelling defence to date is adversarial training and consists of complementing the training data set with adversarial examples. Yet adversarial training severely impacts training time and depends on finding representative adversarial samples. In this paper we propose to train models on output spaces with large class separation in order to gain robustness without adversarial training. We introduce a method to partition the output space into class prototypes with large separation and train models to preserve it. Experimental results shows that models trained with these prototypes -- which we call deep repulsive prototypes -- gain robustness competitive with adversarial training, while also preserving more accuracy on natural samples. Moreover, the models are more resilient to large perturbation sizes. For example, we obtained over 50% robustness for CIFAR-10, with 92% accuracy on natural samples and over 20% robustness for CIFAR-100, with 71% accuracy on natural samples without adversarial training. For both data sets, the models preserved robustness against large perturbations better than adversarially trained models.