Neural network training under semidefinite constraints

TL;DR

This paper introduces an efficient method for training neural networks with semidefinite constraints to ensure robustness and stability, particularly enforcing Lipschitz bounds in large-scale deep networks like WGANs.

Contribution

It presents a scalable interior point method leveraging the banded structure of semidefinite constraints for neural network training with robustness guarantees.

Findings

Method effectively enforces Lipschitz constraints in large neural networks.

Numerical results demonstrate superiority over existing approaches.

Applicable to training Wasserstein GANs with stability guarantees.

Abstract

This paper is concerned with the training of neural networks (NNs) under semidefinite constraints, which allows for NN training with robustness and stability guarantees. In particular, we focus on Lipschitz bounds for NNs. Exploiting the banded structure of the underlying matrix constraint, we set up an efficient and scalable training scheme for NN training problems of this kind based on interior point methods. Our implementation allows to enforce Lipschitz constraints in the training of large-scale deep NNs such as Wasserstein generative adversarial networks (WGANs) via semidefinite constraints. In numerical examples, we show the superiority of our method and its applicability to WGAN training.