Robust Regularization with Adversarial Labelling of Perturbed Samples

TL;DR

This paper introduces ALPS, a novel regularization method that enhances neural network generalization and adversarial robustness by adversarially labeling perturbed synthetic samples, with efficient computation and state-of-the-art results.

Contribution

The paper proposes ALPS, a new regularization scheme based on Vicinal Risk Minimization, with an analytic solution for adversarial labeling that improves robustness and generalization.

Findings

ALPS achieves state-of-the-art regularization performance.

ALPS effectively enhances adversarial robustness.

ALPS performs well across multiple datasets.

Abstract

Recent researches have suggested that the predictive accuracy of neural network may contend with its adversarial robustness. This presents challenges in designing effective regularization schemes that also provide strong adversarial robustness. Revisiting Vicinal Risk Minimization (VRM) as a unifying regularization principle, we propose Adversarial Labelling of Perturbed Samples (ALPS) as a regularization scheme that aims at improving the generalization ability and adversarial robustness of the trained model. ALPS trains neural networks with synthetic samples formed by perturbing each authentic input sample towards another one along with an adversarially assigned label. The ALPS regularization objective is formulated as a min-max problem, in which the outer problem is minimizing an upper-bound of the VRM loss, and the inner problem is L$_1$-ball constrained adversarial labelling on perturbed sample. The analytic solution to the induced inner maximization problem is elegantly derived, which enables computational efficiency. Experiments on the SVHN, CIFAR-10, CIFAR-100 and Tiny-ImageNet datasets show that the ALPS has a state-of-the-art regularization performance while also serving as an effective adversarial training scheme.