Second Order Correctness of Perturbation Bootstrap M-Estimator of Multiple Linear Regression Parameter

TL;DR

This paper demonstrates that a modified perturbation bootstrap method achieves second order correctness in approximating the distribution of M-estimators in multiple linear regression, improving inference accuracy over traditional methods.

Contribution

It introduces a novel modification to the studentized bootstrap estimator that attains second order correctness, even with non-identically distributed errors.

Findings

Classical studentized bootstrap fails to be second order correct.

Modified bootstrap pivot achieves second order correctness.

Perturbation bootstrap remains effective with non-i.i.d. errors.

Abstract

Consider the multiple linear regression model $y_{i} = \boldsymbol{x}'_{i} \boldsymbol{\beta} + \epsilon_{i}$, where $\epsilon_i$'s are independent and identically distributed random variables, $\mathbf{x}_i$'s are known design vectors and $\boldsymbol{\beta}$ is the $p \times 1$ vector of parameters. An effective way of approximating the distribution of the M-estimator $\boldsymbol{\bar{\beta}}_n$, after proper centering and scaling, is the Perturbation Bootstrap Method. In this current work, second order results of this non-naive bootstrap method have been investigated. Second order correctness is important for reducing the approximation error uniformly to $o(n^{-1/2})$ to get better inferences. We show that the classical studentized version of the bootstrapped estimator fails to be second order correct. We introduce an innovative modification in the studentized version of the bootstrapped statistic and show that the modified bootstrapped pivot is second order correct (S.O.C.) for approximating the distribution of the studentized M-estimator. Additionally, we show that the Perturbation Bootstrap continues to be S.O.C. when the errors $\epsilon_i$'s are independent, but may not be identically distributed. These findings establish perturbation Bootstrap approximation as a significant improvement over asymptotic normality in the regression M-estimation.