Provable Robustness of ReLU networks via Maximization of Linear Regions

TL;DR

This paper introduces a regularization method for ReLU neural networks that enhances robustness by increasing linear regions and decision boundary distance, providing provable guarantees and improved adversarial resistance.

Contribution

The paper presents a novel regularization scheme that maximizes linear regions and decision boundary distance, offering provable robustness guarantees for ReLU networks.

Findings

Improves robustness bounds over adversarial training

Allows finding minimal adversarial perturbations for some test points

Achieves comparable or better robustness and test error than state-of-the-art methods

Abstract

It has been shown that neural network classifiers are not robust. This raises concerns about their usage in safety-critical systems. We propose in this paper a regularization scheme for ReLU networks which provably improves the robustness of the classifier by maximizing the linear regions of the classifier as well as the distance to the decision boundary. Our techniques allow even to find the minimal adversarial perturbation for a fraction of test points for large networks. In the experiments we show that our approach improves upon adversarial training both in terms of lower and upper bounds on the robustness and is comparable or better than the state-of-the-art in terms of test error and robustness.