Bagging cross-validated bandwidth selection in nonparametric regression estimation with applications to large-sized samples

TL;DR

This paper proposes a bagging approach to improve cross-validation bandwidth selection in nonparametric regression, reducing variability and computational time, especially for large samples, with theoretical analysis and empirical validation.

Contribution

It introduces a bagging cross-validation method with asymptotic analysis, demonstrating improved convergence rates and efficiency over traditional leave-one-out methods.

Findings

Bagging cross-validation achieves an $n^{-1/2}$ convergence rate.

The method reduces variability and computational time.

Empirical results show better performance on real data.

Abstract

Cross-validation is a well-known and widely used bandwidth selection method in nonparametric regression estimation. However, this technique has two remarkable drawbacks: (i) the large variability of the selected bandwidths, and (ii) the inability to provide results in a reasonable time for very large sample sizes. To overcome these problems, bagging cross-validation bandwidths are analyzed in this paper. This approach consists in computing the cross-validation bandwidths for a finite number of subsamples and then rescaling the averaged smoothing parameters to the original sample size. Under a random-design regression model, asymptotic expressions up to a second-order for the bias and variance of the leave-one-out cross-validation bandwidth for the Nadaraya--Watson estimator are obtained. Subsequently, the asymptotic bias and variance and the limit distribution are derived for the bagged cross-validation selector. Suitable choices of the number of subsamples and the subsample size lead to an $n^{-1/2}$ rate for the convergence in distribution of the bagging cross-validation selector, outperforming the rate $n^{-3/10}$ of leave-one-out cross-validation. Several simulations and an illustration on a real dataset related to the COVID-19 pandemic show the behavior of our proposal and its better performance, in terms of statistical efficiency and computing time, when compared to leave-one-out cross-validation.