Fast Training of Deep Neural Networks Robust to Adversarial Perturbations

TL;DR

This paper extends fast adversarial training methods to Euclidean norm perturbations, demonstrating that it maintains robustness and interpretability while significantly reducing training time through distributed schemes.

Contribution

It introduces a fast approximation for adversarial training under Euclidean norms and shows that distributed training further accelerates robust deep neural network training.

Findings

Fast adversarial training extends to Euclidean norm perturbations.

Distributed training reduces overall training time.

Robust models retain interpretability and human-aligned features.

Abstract

Deep neural networks are capable of training fast and generalizing well within many domains. Despite their promising performance, deep networks have shown sensitivities to perturbations of their inputs (e.g., adversarial examples) and their learned feature representations are often difficult to interpret, raising concerns about their true capability and trustworthiness. Recent work in adversarial training, a form of robust optimization in which the model is optimized against adversarial examples, demonstrates the ability to improve performance sensitivities to perturbations and yield feature representations that are more interpretable. Adversarial training, however, comes with an increased computational cost over that of standard (i.e., nonrobust) training, rendering it impractical for use in large-scale problems. Recent work suggests that a fast approximation to adversarial training shows promise for reducing training time and maintaining robustness in the presence of perturbations bounded by the infinity norm. In this work, we demonstrate that this approach extends to the Euclidean norm and preserves the human-aligned feature representations that are common for robust models. Additionally, we show that using a distributed training scheme can further reduce the time to train robust deep networks. Fast adversarial training is a promising approach that will provide increased security and explainability in machine learning applications for which robust optimization was previously thought to be impractical.