Resampling-free bootstrap inference for quantiles

TL;DR

This paper introduces resampling-free Poisson bootstrap algorithms for quantile inference, significantly reducing computational costs and enabling large-scale applications like Spotify's massive A/B tests.

Contribution

It develops novel, computationally efficient Poisson bootstrap methods for quantile inference that work on very large datasets without additional assumptions.

Findings

Enables bootstrap confidence intervals for quantiles in large samples

Reduces computational complexity of bootstrap inference

Applicable to billions of observations in practice

Abstract

Bootstrap inference is a powerful tool for obtaining robust inference for quantiles and difference-in-quantiles estimators. The computationally intensive nature of bootstrap inference has made it infeasible in large-scale experiments. In this paper, the theoretical properties of the Poisson bootstrap algorithm and quantile estimators are used to derive alternative resampling-free algorithms for Poisson bootstrap inference that reduce the computational complexity substantially without additional assumptions. These findings are connected to existing literature on analytical confidence intervals for quantiles based on order statistics. The results unlock bootstrap inference for difference-in-quantiles for almost arbitrarily large samples. At Spotify, we can now easily calculate bootstrap confidence intervals for quantiles and difference-in-quantiles in A/B tests with hundreds of millions of observations.