On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models

TL;DR

This paper demonstrates that a simple interval bound propagation technique can effectively train large, provably robust neural networks, outperforming complex methods and achieving state-of-the-art verified accuracy on multiple datasets.

Contribution

The authors show how IBP, a straightforward bounding method, can be used with specific loss functions and hyper-parameters to train large robust models efficiently.

Findings

IBP-based training outperforms more complex methods.

Achieved state-of-the-art verified accuracy on MNIST, CIFAR-10, and SVHN.

Trained the largest verified model on a downscaled ImageNet.

Abstract

Recent work has shown that it is possible to train deep neural networks that are provably robust to norm-bounded adversarial perturbations. Most of these methods are based on minimizing an upper bound on the worst-case loss over all possible adversarial perturbations. While these techniques show promise, they often result in difficult optimization procedures that remain hard to scale to larger networks. Through a comprehensive analysis, we show how a simple bounding technique, interval bound propagation (IBP), can be exploited to train large provably robust neural networks that beat the state-of-the-art in verified accuracy. While the upper bound computed by IBP can be quite weak for general networks, we demonstrate that an appropriate loss and clever hyper-parameter schedule allow the network to adapt such that the IBP bound is tight. This results in a fast and stable learning algorithm that outperforms more sophisticated methods and achieves state-of-the-art results on MNIST, CIFAR-10 and SVHN. It also allows us to train the largest model to be verified beyond vacuous bounds on a downscaled version of ImageNet.