What Teachers Should Know about the Bootstrap: Resampling in the Undergraduate Statistics Curriculum

TL;DR

This paper advocates for integrating bootstrap and permutation tests into undergraduate statistics education, emphasizing their advantages over traditional methods, explaining their workings, limitations, and promoting more accurate resampling-based inference.

Contribution

It introduces practical teaching strategies for bootstrap methods, compares their accuracy with classical tests, and highlights their importance in improving statistical understanding and practice.

Findings

Bootstrap enhances understanding of sampling distributions.

Bootstrap percentile intervals underperform in small samples.

Resampling methods offer more accurate inference than traditional t-tests.

Abstract

I have three goals in this article: (1) To show the enormous potential of bootstrapping and permutation tests to help students understand statistical concepts including sampling distributions, standard errors, bias, confidence intervals, null distributions, and P-values. (2) To dig deeper, understand why these methods work and when they don't, things to watch out for, and how to deal with these issues when teaching. (3) To change statistical practice---by comparing these methods to common t tests and intervals, we see how inaccurate the latter are; we confirm this with asymptotics. n >= 30 isn't enough---think n >= 5000. Resampling provides diagnostics, and more accurate alternatives. Sadly, the common bootstrap percentile interval badly under-covers in small samples; there are better alternatives. The tone is informal, with a few stories and jokes.