Globally-Robust Neural Networks

TL;DR

This paper introduces a globally-robust training framework for neural networks that achieves state-of-the-art verifiable accuracy with less computational cost by incorporating global Lipschitz bounds, enabling efficient real-time certification.

Contribution

It formalizes global robustness for neural networks and demonstrates how to adapt existing architectures with global Lipschitz bounds for certifiable robustness and efficiency.

Findings

Achieves state-of-the-art verifiable accuracy with less training time and memory.

Enables real-time certification with negligible additional costs.

Large robust models can be trained in hours using the proposed method.

Abstract

The threat of adversarial examples has motivated work on training certifiably robust neural networks to facilitate efficient verification of local robustness at inference time. We formalize a notion of global robustness, which captures the operational properties of on-line local robustness certification while yielding a natural learning objective for robust training. We show that widely-used architectures can be easily adapted to this objective by incorporating efficient global Lipschitz bounds into the network, yielding certifiably-robust models by construction that achieve state-of-the-art verifiable accuracy. Notably, this approach requires significantly less time and memory than recent certifiable training methods, and leads to negligible costs when certifying points on-line; for example, our evaluation shows that it is possible to train a large robust Tiny-Imagenet model in a matter of hours. Our models effectively leverage inexpensive global Lipschitz bounds for real-time certification, despite prior suggestions that tighter local bounds are needed for good performance; we posit this is possible because our models are specifically trained to achieve tighter global bounds. Namely, we prove that the maximum achievable verifiable accuracy for a given dataset is not improved by using a local bound.