On Uniform Error Bounds for Kernel Regression under Non-Gaussian Noise

TL;DR

This paper introduces new non-asymptotic probabilistic uniform error bounds for kernel regression that accommodate a wide range of non-Gaussian noise types, including correlated noise, enhancing uncertainty quantification in machine learning.

Contribution

It provides the first non-asymptotic bounds for kernel regression applicable to diverse non-Gaussian noise distributions and correlated noise scenarios.

Findings

Bounds are tight compared to existing results.

Applicable to correlated and uncorrelated non-Gaussian noise.

Improve uncertainty quantification in safety-critical applications.

Abstract

Providing non-conservative uncertainty quantification for function estimates derived from noisy observations remains a fundamental challenge in statistical machine learning, particularly for applications in safety-critical domains. In this work, we propose novel non-asymptotic probabilistic uniform error bounds for kernel-based regression. Compared to related bounds in the literature that are restricted to (conditionally) independent sub-Gaussian noise, our bounds allow to consider a broad class of non-Gaussian distributions, such as sub-Gaussian, bounded, sub-exponential, and variance/moment-bounded noise. Moreover, our results apply to correlated and uncorrelated noise. We compare our proposed error bounds with existing results in terms of the induced uncertainty region and their performance in safe control, demonstrating the tightness of the proposed bounds.