Lipschitz Interpolation: Non-parametric Convergence under Bounded Stochastic Noise

TL;DR

This paper analyzes the convergence of Lipschitz interpolation methods under bounded stochastic noise, providing theoretical guarantees, convergence rates, and applications to online learning and control stability.

Contribution

It offers new probabilistic consistency guarantees and convergence bounds for Lipschitz interpolation, extending to online learning and removing the need for known Lipschitz constants.

Findings

Established probabilistic consistency of Lipschitz interpolation.

Derived upper bounds on uniform convergence rates.

Applied results to online learning and control stability.

Abstract

This paper examines the asymptotic convergence properties of Lipschitz interpolation methods within the context of bounded stochastic noise. In the first part of the paper, we establish probabilistic consistency guarantees of the classical approach in a general setting and derive upper bounds on the uniform convergence rates. These bounds align with well-established optimal rates of non-parametric regression obtained in related settings and provide new precise upper bounds on the non-parametric regression problem under bounded noise assumptions. Practically, they can serve as a theoretical tool for comparing Lipschitz interpolation to alternative non-parametric regression methods, providing a condition on the behaviour of the noise at the boundary of its support which indicates when Lipschitz interpolation should be expected to asymptotically outperform or underperform other approaches. In the second part, we expand upon these results to include asymptotic guarantees for online learning of dynamics in discrete-time stochastic systems and illustrate their utility in deriving closed-loop stability guarantees of a simple controller. We also explore applications where the main assumption of prior knowledge of the Lipschitz constant is removed by adopting the LACKI framework (Calliess et al. (2020)) and deriving general asymptotic consistency.