Adaptive Bayesian credible sets in regression with a Gaussian process prior

TL;DR

This paper studies adaptive Bayesian methods for nonparametric regression using Gaussian process priors, demonstrating their ability to adapt to function smoothness and provide credible sets with valid coverage under certain conditions.

Contribution

It compares empirical and hierarchical Bayes approaches, establishing their adaptive contraction rates and coverage properties for credible sets in Gaussian process regression.

Findings

All methods achieve adaptive posterior contraction rates.

Credible sets cover the true function under a self-similarity condition.

Self-similarity condition holds with probability one under the prior.

Abstract

We investigate two empirical Bayes methods and a hierarchical Bayes method for adapting the scale of a Gaussian process prior in a nonparametric regression model. We show that all methods lead to a posterior contraction rate that adapts to the smoothness of the true regression function. Furthermore, we show that the corresponding credible sets cover the true regression function whenever this function satisfies a certain extrapolation condition. This condition depends on the specific method, but is implied by a condition of self-similarity. The latter condition is shown to be satisfied with probability one under the prior distribution.