Lipschitz Networks and Distributional Robustness

TL;DR

This paper establishes a theoretical link between Lipschitz regularization and distributional robustness in deep neural networks, providing bounds on robust risk and insights into adversarial training.

Contribution

It introduces a bound on distributionally robust risk using Lipschitz constants, connecting robustness and regularization in deep neural networks.

Findings

Distributionally robust risk can be bounded by Lipschitz regularization.

Lipschitz constants quantify neural network robustness.

Robust risk bounds relate to adversarial training performance.

Abstract

Robust risk minimisation has several advantages: it has been studied with regards to improving the generalisation properties of models and robustness to adversarial perturbation. We bound the distributionally robust risk for a model class rich enough to include deep neural networks by a regularised empirical risk involving the Lipschitz constant of the model. This allows us to interpretand quantify the robustness properties of a deep neural network. As an application we show the distributionally robust risk upperbounds the adversarial training risk.