Binary sampling from discrete distributions

TL;DR

This paper introduces a binary sampling algorithm for discrete distributions that improves efficiency and accuracy over existing inverse transform sampling methods by utilizing a binary tree structure and parallelizable procedures.

Contribution

The paper proposes a novel binary sampling algorithm with two procedures, BBS and FBS, offering better parallelization and accuracy compared to traditional binary-search ITS methods.

Findings

BBS runs in O(N) time, FBS in O(ln N) time.

The algorithm has O(N) space complexity.

It outperforms standard binary-search ITS in accuracy and parallelizability.

Abstract

This paper considers direct sampling methods from discrete target distributions. The inverse transform sampling (ITS) method is one of the most popular direct sampling methods. The main purpose of this paper is to propose a direct sampling algorithm that supersedes the binary-search ITS method (which is an improvement of the ITS method with binary search). The proposed algorithm is based on binarizing the support set of the target distribution. Thus, the proposed algorithm is referred to as binary sampling (BS). The BS algorithm consists of two procedures: backward binary sampling (BBS) and forward binary sampling (FBS). The BBS procedure draws a single sample (the first sample) from the target distribution while constructing a one-way random walk on a binary tree for the FBS procedure. By running the random walk, the FBS procedure generates the second and subsequent samples. The BBS and FBS procedures have $O(N)$ and $O(\ln N)$ time complexities, respectively, and they also have $O(N)$ space complexity, where $N+1$ is the cardinality of the support set of the target distribution. Therefore, the time and space complexities of the BS algorithm are equivalent to those of the standard (possibly best) binary-search ITS algorithm. However, the BS algorithm has two advantages over the standard binary-search ITS algorithm. First, the BBS procedure is parallelizable and thus the total running time of the BS algorithm can be reduced. Second, the BS algorithm is more accurate in terms of relative rounding error that influences generated samples.