Lipschitz regularized Deep Neural Networks generalize and are adversarially robust

TL;DR

This paper shows that Lipschitz regularization of deep neural networks enhances their generalization and adversarial robustness, with theoretical proofs and empirical evidence supporting its effectiveness and potential for attack detection.

Contribution

It introduces a novel regularization approach combining adversarial training and Lipschitz constraints, providing theoretical insights and empirical validation for improved robustness.

Findings

Regularized models exhibit increased adversarial robustness.

Gradient norms can be used effectively for attack detection.

Generalization bounds are independent of network depth.

Abstract

In this work we study input gradient regularization of deep neural networks, and demonstrate that such regularization leads to generalization proofs and improved adversarial robustness. The proof of generalization does not overcome the curse of dimensionality, but it is independent of the number of layers in the networks. The adversarial robustness regularization combines adversarial training, which we show to be equivalent to Total Variation regularization, with Lipschitz regularization. We demonstrate empirically that the regularized models are more robust, and that gradient norms of images can be used for attack detection.