Adversarial Margin Maximization Networks

TL;DR

This paper introduces adversarial margin maximization (AMM), a regularization technique that enhances DNN generalization and robustness by encouraging larger input space margins through adversarial perturbations.

Contribution

It proposes a novel regularization method, AMM, that uses adversarial perturbations to improve DNN generalization and robustness from a geometric perspective.

Findings

AMM improves model robustness against adversarial attacks.

AMM outperforms previous methods on multiple datasets.

The approach is simple to implement and train end-to-end.

Abstract

The tremendous recent success of deep neural networks (DNNs) has sparked a surge of interest in understanding their predictive ability. Unlike the human visual system which is able to generalize robustly and learn with little supervision, DNNs normally require a massive amount of data to learn new concepts. In addition, research works also show that DNNs are vulnerable to adversarial examples-maliciously generated images which seem perceptually similar to the natural ones but are actually formed to fool learning models, which means the models have problem generalizing to unseen data with certain type of distortions. In this paper, we analyze the generalization ability of DNNs comprehensively and attempt to improve it from a geometric point of view. We propose adversarial margin maximization (AMM), a learning-based regularization which exploits an adversarial perturbation as a proxy. It encourages a large margin in the input space, just like the support vector machines. With a differentiable formulation of the perturbation, we train the regularized DNNs simply through back-propagation in an end-to-end manner. Experimental results on various datasets (including MNIST, CIFAR-10/100, SVHN and ImageNet) and different DNN architectures demonstrate the superiority of our method over previous state-of-the-arts. Code and models for reproducing our results will be made publicly available.