Bayesian bandwidth estimation for local linear fitting in nonparametric regression models

TL;DR

This paper introduces a Bayesian sampling method for estimating bandwidths in local linear regression, improving accuracy over traditional methods by jointly estimating error density and bandwidths.

Contribution

It extends Bayesian bandwidth estimation to local linear estimators with kernel-form error density, providing a data-driven approach for simultaneous bandwidth selection.

Findings

Outperforms rule-of-thumb and cross-validation methods in simulations

Provides more accurate estimation of regression functions

Validated on real-world nonparametric regression models

Abstract

This paper presents a Bayesian sampling approach to bandwidth estimation for the local linear estimator of the regression function in a nonparametric regression model. In the Bayesian sampling approach, the error density is approximated by a location-mixture density of Gaussian densities with means the individual errors and variance a constant parameter. This mixture density has the form of a kernel density estimator of errors and is referred to as the kernel-form error density (c.f., Zhang et al., 2014). While Zhang et al. (2014) use the local constant (also known as the Nadaraya- Watson) estimator to estimate the regression function, we extend this to the local linear estimator, which produces more accurate estimation. The proposed investigation is motivated by the lack of data-driven methods for simultaneously choosing bandwidths in the local linear estimator of the regression function and kernel-form error density. Treating bandwidths as parameters, we derive an approximate (pseudo) likelihood and a posterior. A simulation study shows that the proposed bandwidth estimation outperforms the rule-of-thumb and cross-validation methods under the criterion of integrated squared errors. The proposed bandwidth estimation method is validated through a nonparametric regression model involving firm ownership concentration, and a model involving state-price density estimation.