Fast Training of Provably Robust Neural Networks by SingleProp

TL;DR

This paper introduces SingleProp, a new regularizer that enables faster training of neural networks with certified robustness guarantees, maintaining accuracy while reducing computational costs.

Contribution

The paper proposes SingleProp, a regularizer that significantly speeds up training of robust neural networks with minimal loss in certified accuracy.

Findings

Faster training times on MNIST and CIFAR-10

Comparable certified robustness accuracy to existing methods

Reduced computational cost per training iteration

Abstract

Recent works have developed several methods of defending neural networks against adversarial attacks with certified guarantees. However, these techniques can be computationally costly due to the use of certification during training. We develop a new regularizer that is both more efficient than existing certified defenses, requiring only one additional forward propagation through a network, and can be used to train networks with similar certified accuracy. Through experiments on MNIST and CIFAR-10 we demonstrate improvements in training speed and comparable certified accuracy compared to state-of-the-art certified defenses.