A Dynamical System Perspective for Lipschitz Neural Networks

TL;DR

This paper introduces a novel framework for constructing 1-Lipschitz neural networks using a dynamical systems approach, including the Convex Potential Layer, to enhance robustness against adversarial attacks.

Contribution

It provides a generic method to build 1-Lipschitz networks from residual networks and introduces the Convex Potential Layer, unifying previous approaches and improving adversarial robustness.

Findings

The proposed architecture is scalable to multiple datasets.

The method offers provable $ ext{L}_2$ robustness against adversarial examples.

Previous approaches are special cases within the proposed framework.

Abstract

The Lipschitz constant of neural networks has been established as a key quantity to enforce the robustness to adversarial examples. In this paper, we tackle the problem of building $1$-Lipschitz Neural Networks. By studying Residual Networks from a continuous time dynamical system perspective, we provide a generic method to build $1$-Lipschitz Neural Networks and show that some previous approaches are special cases of this framework. Then, we extend this reasoning and show that ResNet flows derived from convex potentials define $1$-Lipschitz transformations, that lead us to define the {\em Convex Potential Layer} (CPL). A comprehensive set of experiments on several datasets demonstrates the scalability of our architecture and the benefits as an $\ell_2$-provable defense against adversarial examples.