Spectral regularization for adversarially-robust representation learning

TL;DR

This paper introduces a spectral regularizer designed to enhance adversarial robustness in neural network representations, especially effective in self-supervised and transfer learning scenarios, outperforming previous methods.

Contribution

The paper proposes a novel spectral regularizer that specifically targets feature representations to improve adversarial robustness, particularly in self-supervised and transfer learning contexts.

Findings

Spectral regularization improves test accuracy and robustness in supervised learning.

The method enhances adversarial robustness of self-supervised and transferred representations.

It reveals how representational structure influences adversarial robustness.

Abstract

The vulnerability of neural network classifiers to adversarial attacks is a major obstacle to their deployment in safety-critical applications. Regularization of network parameters during training can be used to improve adversarial robustness and generalization performance. Usually, the network is regularized end-to-end, with parameters at all layers affected by regularization. However, in settings where learning representations is key, such as self-supervised learning (SSL), layers after the feature representation will be discarded when performing inference. For these models, regularizing up to the feature space is more suitable. To this end, we propose a new spectral regularizer for representation learning that encourages black-box adversarial robustness in downstream classification tasks. In supervised classification settings, we show empirically that this method is more effective in boosting test accuracy and robustness than previously-proposed methods that regularize all layers of the network. We then show that this method improves the adversarial robustness of classifiers using representations learned with self-supervised training or transferred from another classification task. In all, our work begins to unveil how representational structure affects adversarial robustness.