Adaptive Bayesian nonparametric regression using kernel mixture of polynomials with application to partial linear model

TL;DR

This paper introduces a Bayesian nonparametric regression method using kernel mixtures of polynomials, achieving adaptive, minimax-optimal convergence rates and effective application to partial linear models with superior empirical performance.

Contribution

It develops a novel kernel mixture of polynomials prior that adapts to unknown smoothness levels and applies it to partial linear models, providing theoretical and practical advancements.

Findings

Achieves minimax-optimal posterior contraction rates up to a logarithmic factor.

Provides a near-optimal sieve maximum likelihood estimator.

Demonstrates superior performance in simulations and real data applications.

Abstract

We propose a kernel mixture of polynomials prior for Bayesian nonparametric regression. The regression function is modeled by local averages of polynomials with kernel mixture weights. We obtain the minimax-optimal rate of contraction of the full posterior distribution up to a logarithmic factor that adapts to the smoothness level of the true function by estimating metric entropies of certain function classes. We also provide a frequentist sieve maximum likelihood estimator with a near-optimal convergence rate. We further investigate the application of the kernel mixture of polynomials to the partial linear model and obtain both the near-optimal rate of contraction for the nonparametric component and the Bernstein-von Mises limit (i.e., asymptotic normality) of the parametric component. The proposed method is illustrated with numerical examples and shows superior performance in terms of computational efficiency, accuracy, and uncertainty quantification compared to the local polynomial regression, DiceKriging, and the robust Gaussian stochastic process.