Exact and Efficient Sampling from Dynamic Discrete Distributions with Finite-Precision Weights

TL;DR

This paper introduces EBUS, an exact and efficient dynamic sampling method for finite-precision weights, ensuring precise probabilities and outperforming previous inexact approaches.

Contribution

EBUS is the first method providing exact sampling with finite-precision weights, supporting constant-time sampling and updates in a practical computational model.

Findings

EBUS guarantees exact probability proportionality for each sampled index.

The method achieves O(1) worst-case expected sampling time and O(1) amortized update time.

Experimental results show EBUS is competitive with or faster than previous inexact methods.

Abstract

Sampling from a dynamic discrete distribution means drawing an index with probability proportional to a mutable set of weights. Classical constant-time techniques such as the Alias Method are well suited to static distributions, but become expensive in dynamic settings because updates require rebuilding auxiliary tables. Existing dynamic approaches, including Forest of Trees and BUcket Sampling (BUS), achieve reasonable practical performance but require infinite precision real arithmetic to be correct and produce meaningfully incorrect results when implemented on real hardware. We present EBUS (Exact BUcket Sampling), a dynamic sampler for finite-precision weights that is exact by construction: every returned index has probability exactly proportional to its represented weight. Our guarantees are proved in a word RAM model with bounded exponent range. In that model, our method supports $O(1)$ worst-case expected sampling time, $O(1)$ amortized time to update a single weight, $O(n)$ space, and $O(n)$ construction. We also provide an implementation for IEEE 64-bit floating-point weights and show experimentally that it is competitive with, and often faster than, several implementations of previous inexact methods while avoiding their numerical failure modes.