Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization

TL;DR

This paper introduces a robust optimization framework for adversarial training of neural networks, enhancing their local stability and robustness against adversarial examples while also improving test accuracy.

Contribution

It presents a general alternating minimization-maximization framework that unifies and extends previous adversarial training methods for neural networks.

Findings

Increases neural network robustness to adversarial examples.

Makes it harder to generate new adversarial examples.

Improves accuracy on original test data.

Abstract

We propose a general framework for increasing local stability of Artificial Neural Nets (ANNs) using Robust Optimization (RO). We achieve this through an alternating minimization-maximization procedure, in which the loss of the network is minimized over perturbed examples that are generated at each parameter update. We show that adversarial training of ANNs is in fact robustification of the network optimization, and that our proposed framework generalizes previous approaches for increasing local stability of ANNs. Experimental results reveal that our approach increases the robustness of the network to existing adversarial examples, while making it harder to generate new ones. Furthermore, our algorithm improves the accuracy of the network also on the original test data.