TL;DR
This paper introduces a simple, scalable, and cache-efficient divide-and-conquer algorithm for random sampling without replacement that performs well on modern parallel architectures, including GPUs and SIMD units.
Contribution
It presents a novel parallel and cache-efficient sampling algorithm with minimal communication, suitable for online and vectorized implementations on modern hardware.
Findings
Expected time complexity is O(n/p + log p) on p processors.
Minimal communication overhead of O(log p), independent of sample size.
Algorithm adapts to various sampling modes and hardware architectures.
Abstract
We consider the problem of sampling numbers from the range without replacement on modern architectures. The main result is a simple divide-and-conquer scheme that makes sequential algorithms more cache efficient and leads to a parallel algorithm running in expected time on processors, i.e., scales to massively parallel machines even for moderate values of . The amount of communication between the processors is very small (at most ) and independent of the sample size. We also discuss modifications needed for load balancing, online sampling, sampling with replacement, Bernoulli sampling, and vectorization on SIMD units or GPUs.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
