Anisotropic function estimation using multi-bandwidth Gaussian processes

TL;DR

This paper introduces a Bayesian Gaussian process method for anisotropic multivariate regression that adapts to unknown smoothness and dimension, achieving near-minimax optimal rates.

Contribution

It proposes a multi-bandwidth Gaussian process prior with hyperpriors that adaptively estimates anisotropic surfaces, improving over single-bandwidth approaches.

Findings

Achieves near-minimax posterior contraction rates

Demonstrates sub-optimality of single bandwidth Gaussian processes in anisotropic settings

Provides a theoretically justified Bayesian procedure for anisotropic function estimation

Abstract

In nonparametric regression problems involving multiple predictors, there is typically interest in estimating an anisotropic multivariate regression surface in the important predictors while discarding the unimportant ones. Our focus is on defining a Bayesian procedure that leads to the minimax optimal rate of posterior contraction (up to a log factor) adapting to the unknown dimension and anisotropic smoothness of the true surface. We propose such an approach based on a Gaussian process prior with dimension-specific scalings, which are assigned carefully-chosen hyperpriors. We additionally show that using a homogenous Gaussian process with a single bandwidth leads to a sub-optimal rate in anisotropic cases.