SOAR: Second-Order Adversarial Regularization

TL;DR

This paper introduces SOAR, a second-order adversarial regularizer derived from a Taylor series approximation, which enhances neural network robustness against $ ext{L}_ ext{infty}$ and $ ext{L}_ ext{2}$ adversarial attacks.

Contribution

The paper proposes a novel second-order regularizer for adversarial training, providing a new theoretical approach to improve robustness of neural networks.

Findings

Significantly improves robustness against $ ext{L}_ ext{infty}$ and $ ext{L}_ ext{2}$ attacks.

Effective on CIFAR-10 and SVHN datasets.

Based on a Taylor series approximation of the adversarial loss.

Abstract

Adversarial training is a common approach to improving the robustness of deep neural networks against adversarial examples. In this work, we propose a novel regularization approach as an alternative. To derive the regularizer, we formulate the adversarial robustness problem under the robust optimization framework and approximate the loss function using a second-order Taylor series expansion. Our proposed second-order adversarial regularizer (SOAR) is an upper bound based on the Taylor approximation of the inner-max in the robust optimization objective. We empirically show that the proposed method significantly improves the robustness of networks against the $\ell_\infty$ and $\ell_2$ bounded perturbations generated using cross-entropy-based PGD on CIFAR-10 and SVHN.