Towards Robust Deep Neural Networks with BANG

TL;DR

This paper introduces BANG, a novel training method that enhances the robustness of deep neural networks against adversarial perturbations without relying on data augmentation or adversarial examples.

Contribution

The paper presents a new theory explaining neural network vulnerabilities and proposes BANG, an efficient training approach that improves robustness while maintaining or improving accuracy.

Findings

BANG significantly increases model robustness to adversarial attacks.

Models trained with BANG outperform baseline models in robustness tests.

BANG does not require adversarial data augmentation or adversarial training.

Abstract

Machine learning models, including state-of-the-art deep neural networks, are vulnerable to small perturbations that cause unexpected classification errors. This unexpected lack of robustness raises fundamental questions about their generalization properties and poses a serious concern for practical deployments. As such perturbations can remain imperceptible - the formed adversarial examples demonstrate an inherent inconsistency between vulnerable machine learning models and human perception - some prior work casts this problem as a security issue. Despite the significance of the discovered instabilities and ensuing research, their cause is not well understood and no effective method has been developed to address the problem. In this paper, we present a novel theory to explain why this unpleasant phenomenon exists in deep neural networks. Based on that theory, we introduce a simple, efficient, and effective training approach, Batch Adjusted Network Gradients (BANG), which significantly improves the robustness of machine learning models. While the BANG technique does not rely on any form of data augmentation or the utilization of adversarial images for training, the resultant classifiers are more resistant to adversarial perturbations while maintaining or even enhancing the overall classification performance.