On the Convergence of Bayesian Regression Models

TL;DR

This paper analyzes the convergence properties of Bayesian heteroscedastic nonparametric regression models, providing theoretical rates for posterior distributions with various priors, applicable to both random and deterministic covariates.

Contribution

It establishes adaptive convergence rates for Bayesian nonparametric regression models with different priors, extending existing theory to heteroscedastic settings and both covariate types.

Findings

Derived convergence rates for posterior distributions with splines and Gaussian process priors.

Results applicable to all regularity levels, demonstrating adaptivity.

Theoretical framework accommodating both random and deterministic covariates.

Abstract

We consider heteroscedastic nonparametric regression models, when both the mean function and variance function are unknown and to be estimated with nonparametric approaches. We derive convergence rates of posterior distributions for this model with different priors, including splines and Gaussian process priors. The results are based on the general ones on the rates of convergence of posterior distributions for independent, non-identically distributed observations, and are established for both of the cases with random covariates, and deterministic covariates. We also illustrate that the results can be achieved for all levels of regularity, which means they are adaptive.