An Orthogonal Classifier for Improving the Adversarial Robustness of Neural Networks

TL;DR

This paper introduces a novel orthogonal classifier with dense weight matrices to enhance neural network robustness against adversarial attacks, achieving high accuracy and robustness without complex modifications.

Contribution

The paper proposes a new orthogonal classifier with dense weight matrices that improves adversarial robustness and avoids structural redundancy issues in neural networks.

Findings

Achieves high accuracy on clean data.

Provides improved robustness with adversarial samples.

Competitive performance compared to state-of-the-art methods.

Abstract

Neural networks are susceptible to artificially designed adversarial perturbations. Recent efforts have shown that imposing certain modifications on classification layer can improve the robustness of the neural networks. In this paper, we explicitly construct a dense orthogonal weight matrix whose entries have the same magnitude, thereby leading to a novel robust classifier. The proposed classifier avoids the undesired structural redundancy issue in previous work. Applying this classifier in standard training on clean data is sufficient to ensure the high accuracy and good robustness of the model. Moreover, when extra adversarial samples are used, better robustness can be further obtained with the help of a special worst-case loss. Experimental results show that our method is efficient and competitive to many state-of-the-art defensive approaches. Our code is available at \url{https://github.com/MTandHJ/roboc}.