Deterministic error bounds for kernel-based learning techniques under bounded noise

TL;DR

This paper derives deterministic, finite-sample error bounds for kernel ridge regression and support vector regression when reconstructing functions from bounded noise samples, linking non-parametric learning with system identification.

Contribution

It provides the first deterministic error bounds for these kernel methods under bounded noise, connecting kernel learning with robust control and system identification.

Findings

Finite-sample error bounds established for kernel ridge regression and support vector regression.

Connections drawn between kernel methods and Gaussian processes.

Numerical examples demonstrate the bounds' applicability.

Abstract

We consider the problem of reconstructing a function from a finite set of noise-corrupted samples. Two kernel algorithms are analyzed, namely kernel ridge regression and $\varepsilon$-support vector regression. By assuming the ground-truth function belongs to the reproducing kernel Hilbert space of the chosen kernel, and the measurement noise affecting the dataset is bounded, we adopt an approximation theory viewpoint to establish \textit{deterministic}, finite-sample error bounds for the two models. Finally, we discuss their connection with Gaussian processes and two numerical examples are provided. In establishing our inequalities, we hope to help bring the fields of non-parametric kernel learning and system identification for robust control closer to each other.