Asymptotics of the Empirical Bootstrap Method Beyond Asymptotic Normality

TL;DR

This paper investigates the theoretical properties of the empirical bootstrap method for non-normal estimators, establishing its limiting distribution, consistency conditions, and convergence rate, with practical confidence interval methods.

Contribution

It provides new theoretical insights into the empirical bootstrap's behavior beyond asymptotic normality, including limiting distribution, consistency conditions, and alternative confidence interval constructions.

Findings

Established the limiting distribution of the bootstrap estimator.

Derived tight conditions for bootstrap consistency.

Proposed three new bootstrap-based confidence interval methods.

Abstract

One of the most commonly used methods for forming confidence intervals for statistical inference is the empirical bootstrap, which is especially expedient when the limiting distribution of the estimator is unknown. However, despite its ubiquitous role, its theoretical properties are still not well understood for non-asymptotically normal estimators. In this paper, under stability conditions, we establish the limiting distribution of the empirical bootstrap estimator, derive tight conditions for it to be asymptotically consistent, and quantify the speed of convergence. Moreover, we propose three alternative ways to use the bootstrap method to build confidence intervals with coverage guarantees. Finally, we illustrate the generality and tightness of our results by a series of examples, including uniform confidence bands, two-sample kernel tests, minmax stochastic programs and the empirical risk of stacked estimators.