Bootstrap of residual processes in regression: to smooth or not to smooth ?

TL;DR

This paper proves that the classical non-smooth residual bootstrap method is asymptotically valid for regression models, resolving an open question and demonstrating its effectiveness through simulations.

Contribution

It establishes the asymptotic validity of the non-smooth residual bootstrap in regression, a previously unresolved issue.

Findings

Non-smooth residual bootstrap is asymptotically consistent.

Simulation results confirm bootstrap accuracy across models and sample sizes.

Provides theoretical foundation for using non-smooth bootstrap in regression analysis.

Abstract

In this paper we consider a location model of the form $Y = m(X) + \varepsilon$, where $m(\cdot)$ is the unknown regression function, the error $\varepsilon$ is independent of the $p$-dimensional covariate $X$ and $E(\varepsilon)=0$. Given i.i.d. data $(X_1,Y_1),\ldots,(X_n,Y_n)$ and given an estimator $\hat m(\cdot)$ of the function $m(\cdot)$ (which can be parametric or nonparametric of nature), we estimate the distribution of the error term $\varepsilon$ by the empirical distribution of the residuals $Y_i-\hat m(X_i)$, $i=1,\ldots,n$. To approximate the distribution of this estimator, Koul and Lahiri (1994) and Neumeyer (2008, 2009) proposed bootstrap procedures, based on smoothing the residuals either before or after drawing bootstrap samples. So far it has been an open question whether a classical non-smooth residual bootstrap is asymptotically valid in this context. In this paper we solve this open problem, and show that the non-smooth residual bootstrap is consistent. We illustrate this theoretical result by means of simulations, that show the accuracy of this bootstrap procedure for various models, testing procedures and sample sizes.