Generalised Lipschitz Regularisation Equals Distributional Robustness

TL;DR

This paper establishes a general theoretical link between distributional robustness and Lipschitz regularisation, providing certification methods and new insights into adversarial learning and kernel classifiers.

Contribution

It presents a broad equality result connecting distributional robustness with regularisation, applicable to complex models like universal approximators.

Findings

Certifies robustness of Lipschitz-regularised models under mild assumptions

Reveals a theoretical connection between adversarial learning and distributional robustness

Provides new methods for Lipschitz regularisation of kernel classifiers

Abstract

The problem of adversarial examples has highlighted the need for a theory of regularisation that is general enough to apply to exotic function classes, such as universal approximators. In response, we give a very general equality result regarding the relationship between distributional robustness and regularisation, as defined with a transportation cost uncertainty set. The theory allows us to (tightly) certify the robustness properties of a Lipschitz-regularised model with very mild assumptions. As a theoretical application we show a new result explicating the connection between adversarial learning and distributional robustness. We then give new results for how to achieve Lipschitz regularisation of kernel classifiers, which are demonstrated experimentally.