A Cheap Bootstrap Method for Fast Inference

TL;DR

This paper introduces a computationally efficient bootstrap method that requires only a single resampling per inference task, enabling fast statistical inference for large-scale models without sacrificing accuracy.

Contribution

It proposes a novel bootstrap approach with minimal resampling, extending to nested and subsampling problems, offering a practical solution for fast inference in large models.

Findings

The method achieves reliable statistical guarantees with only one Monte Carlo replication.

It generalizes to nested sampling and subsampling scenarios.

Demonstrates effectiveness across various estimation problems.

Abstract

The bootstrap is a versatile inference method that has proven powerful in many statistical problems. However, when applied to modern large-scale models, it could face substantial computation demand from repeated data resampling and model fitting. We present a bootstrap methodology that uses minimal computation, namely with a resample effort as low as one Monte Carlo replication, while maintaining desirable statistical guarantees. We present the theory of this method that uses a twisted perspective from the standard bootstrap principle. We also present generalizations of this method to nested sampling problems and to a range of subsampling variants, and illustrate how it can be used for fast inference across different estimation problems.