Quantifying Uncertainty: All We Need is the Bootstrap?

TL;DR

This paper demonstrates through review and simulations that the non-parametric bootstrap, especially the double bootstrap, is a simple yet effective universal method for quantifying uncertainty in statistical estimates, outperforming traditional techniques.

Contribution

It provides comprehensive evidence that the double bootstrap outperforms BCa and other methods, advocating for its broader adoption in statistical practice.

Findings

Double bootstrap consistently outperforms BCa in simulations.

Bootstrap methods are viable for basic estimation tasks.

Results support using bootstrap as a universal uncertainty quantification tool.

Abstract

A critical literature review and comprehensive simulation study is used to show that (a) non-parametric bootstrap is a viable alternative to commonly taught and used methods in basic estimation tasks (mean, variance, quartiles, correlation) and (b), contrary to recommendations in most related work, double bootstrap performs better than BCa. Quantifying uncertainty through standard errors, confidence intervals, hypothesis tests, and related measures is a fundamental aspect of statistical practice. However, these techniques involve a variety of methods, mathematical formulas, and underlying concepts, which can be complex. Could the non-parametric bootstrap, known for its simplicity and general applicability, serve as a universal alternative? This paper addresses this question through a review of the existing literature and a simulation analysis of one- and two-sided confidence intervals across varying sample sizes, confidence levels, data-generating processes, and statistical functionals. Results show that the double bootstrap consistently performs best and is a promising alternative to traditional methods used for common statistical tasks. These results suggest that the bootstrap, particularly the double bootstrap, could simplify statistical education and practice without compromising effectiveness.