Rates of contraction of posterior distributions based on Gaussian process priors

TL;DR

This paper establishes how quickly posterior distributions converge in nonparametric Bayesian models using Gaussian process priors, depending on the true parameter's relation to the prior's RKHS and Gaussian process properties.

Contribution

It provides explicit rates of contraction for Gaussian process-based posteriors across various statistical models and prior choices, linking these rates to Gaussian process characteristics.

Findings

Rates depend on the true parameter's position relative to the RKHS.

Explicit rates are derived for models including density estimation, regression, and classification.

Results include analysis of fractional Brownian motion priors and logistic/probit models.

Abstract

We derive rates of contraction of posterior distributions on nonparametric or semiparametric models based on Gaussian processes. The rate of contraction is shown to depend on the position of the true parameter relative to the reproducing kernel Hilbert space of the Gaussian process and the small ball probabilities of the Gaussian process. We determine these quantities for a range of examples of Gaussian priors and in several statistical settings. For instance, we consider the rate of contraction of the posterior distribution based on sampling from a smooth density model when the prior models the log density as a (fractionally integrated) Brownian motion. We also consider regression with Gaussian errors and smooth classification under a logistic or probit link function combined with various priors.