A Theoretical Framework for Bayesian Nonparametric Regression

TL;DR

This paper introduces a flexible Bayesian nonparametric regression framework that unifies various models and provides theoretical results on the rates of posterior contraction for different function classes.

Contribution

It develops a general theoretical framework for Bayesian nonparametric regression and demonstrates its application to multiple models with adaptive and near-optimal contraction rates.

Findings

Adaptive contraction rates for finite random series regression

Exact contraction rates for block prior regression

Near-optimal contraction for Gaussian spline regression

Abstract

We develop a unifying framework for Bayesian nonparametric regression to study the rates of contraction with respect to the integrated $L_2$-distance without assuming the regression function space to be uniformly bounded. The framework is very flexible and can be applied to a wide class of nonparametric prior models. Three non-trivial applications of the proposed framework are provided: The finite random series regression of an $\alpha$-H\"older function, with adaptive rates of contraction up to a logarithmic factor; The un-modified block prior regression of an $\alpha$-Sobolev function, with adaptive-and-exact rates of contraction; The Gaussian spline regression of an $\alpha$-H\"older function, with the near-optimal posterior contraction. These applications serve as generalization or complement of their respective results in the literature. Extensions to the fixed-design regression problem and sparse additive models in high dimensions are discussed as well.