Random Sampling: Practice Makes Imperfect

TL;DR

This paper highlights the inaccuracies in common pseudo-random number generators and sampling algorithms used in statistical software, advocating for cryptographically secure PRNGs and improved methods for better statistical validity.

Contribution

It identifies the limitations of current PRNGs and sampling algorithms, and recommends adopting cryptographically secure PRNGs and more accurate methods for statistical sampling.

Findings

Common PRNGs have limited state spaces causing inaccuracies.

Many sampling algorithms wrongly assume IID uniform outputs.

Cryptographically secure PRNGs improve statistical accuracy.

Abstract

The pseudo-random number generators (PRNGs), sampling algorithms, and algorithms for generating random integers in some common statistical packages and programming languages are unnecessarily inaccurate, by an amount that may matter for statistical inference. Most use PRNGs with state spaces that are too small for contemporary sampling problems and methods such as the bootstrap and permutation tests. The random sampling algorithms in many packages rely on the false assumption that PRNGs produce IID $U[0, 1)$ outputs. The discreteness of PRNG outputs and the limited state space of common PRNGs cause those algorithms to perform poorly in practice. Statistics packages and scientific programming languages should use cryptographically secure PRNGs by default (not for their security properties, but for their statistical ones), and offer weaker PRNGs only as an option. Software should not use methods that assume PRNG outputs are IID $U[0,1)$ random variables, such as generating a random sample by permuting the population and taking the first $k$ items or generating random integers by multiplying a pseudo-random binary fraction or float by a constant and rounding the result. More accurate methods are available.