Practical Global and Local Bounds in Gaussian Process Regression via Chaining

TL;DR

This paper introduces a chaining-based framework for deriving global and local uncertainty bounds in Gaussian process regression that do not depend on specific input features or hyperparameter tuning, improving robustness and tightness.

Contribution

The authors develop a novel chaining-based method for global and local uncertainty bounds in GPR, applicable without feature access and with kernel-specific refinements for tighter estimates.

Findings

Bounds are tighter than generic constructions.

Method outperforms existing approaches on synthetic and real datasets.

Provides both global and local uncertainty quantification.

Abstract

Gaussian process regression (GPR) is a popular nonparametric Bayesian method that provides predictive uncertainty estimates and is widely used in safety-critical applications. While prior research has introduced various uncertainty bounds, most existing approaches require access to specific input features, and rely on posterior mean and variance estimates or the tuning of hyperparameters. These limitations hinder robustness and fail to capture the model's global behavior in expectation. To address these limitations, we propose a chaining-based framework for estimating upper and lower bounds on the expected extreme values over unseen data, without requiring access to specific input features. We provide kernel-specific refinements for commonly used kernels such as RBF and Mat\'ern, in which our bounds are tighter than generic constructions. We further improve numerical tightness by avoiding analytical relaxations. In addition to global estimation, we also develop a novel method for local uncertainty quantification at specified inputs. This approach leverages chaining geometry through partition diameters, adapting to local structures without relying on posterior variance scaling. Our experimental results validate the theoretical findings and demonstrate that our method outperforms existing approaches on both synthetic and real-world datasets.