Remarks on Lipschitz-Minimal Interpolation: Generalization Bounds and Neural Network Implementation

TL;DR

This paper introduces a theoretical framework for Lipschitz-minimal interpolation, providing generalization bounds for neural networks and proposing a neural network implementation with Lipschitz constraints, demonstrated on system dynamics learning.

Contribution

It offers a new approach to function approximation using minimal Lipschitz constant interpolants, with rigorous generalization bounds and a neural network implementation.

Findings

Established generalization bounds for Lipschitz-minimal interpolants.

Developed a neural network implementation with Lipschitz-bounded layers.

Demonstrated the approach on learning system dynamics with certified error bounds.

Abstract

This note establishes a theoretical framework for finding (potentially overparameterized) approximations of a function on a compact set with a-priori bounds for the generalization error. The approximation method considered is to choose, among all functions that (approximately) interpolate a given data set, one with a minimal Lipschitz constant. The paper establishes rigorous generalization bounds over practically relevant classes of approximators, including deep neural networks. It also presents a neural network implementation based on Lipschitz-bounded network layers and an augmented Lagrangian method. The results are illustrated for a problem of learning the dynamics of an input-to-state stable system with certified bounds on simulation error.