On Model Robustness Against Adversarial Examples

TL;DR

This paper introduces a new theoretical framework based on energy functions to analyze and improve the robustness of deep learning models against adversarial examples, outperforming existing methods on multiple datasets.

Contribution

The paper proposes a novel energy-based theory for model robustness and develops a new regularization method that enhances adversarial resistance beyond traditional training approaches.

Findings

Energy regularization improves robustness against adversarial attacks.

The proposed method outperforms previous adversarial training techniques.

Models trained with energy regularization show superior performance on MNIST, CIFAR-10, and SVHN.

Abstract

We study the model robustness against adversarial examples, referred to as small perturbed input data that may however fool many state-of-the-art deep learning models. Unlike previous research, we establish a novel theory addressing the robustness issue from the perspective of stability of the loss function in the small neighborhood of natural examples. We propose to exploit an energy function to describe the stability and prove that reducing such energy guarantees the robustness against adversarial examples. We also show that the traditional training methods including adversarial training with the $l_2$ norm constraint (AT) and Virtual Adversarial Training (VAT) tend to minimize the lower bound of our proposed energy function. We make an analysis showing that minimization of such lower bound can however lead to insufficient robustness within the neighborhood around the input sample. Furthermore, we design a more rational method with the energy regularization which proves to achieve better robustness than previous methods. Through a series of experiments, we demonstrate the superiority of our model on both supervised tasks and semi-supervised tasks. In particular, our proposed adversarial framework achieves the best performance compared with previous adversarial training methods on benchmark datasets MNIST, CIFAR-10, and SVHN. Importantly, they demonstrate much better robustness against adversarial examples than all the other comparison methods.