ASSESSING THE STOCHASTIC PROPERTIES OF MODERN PSEUDO-RANDOM GENERATORS FOR PARALLEL COMPUTING

TL;DR

This paper evaluates the statistical performance and repeatability of various modern pseudo-random number generators used in high performance computing and AI, using extensive testing with over 4.5 years of computational effort.

Contribution

It systematically assesses multiple PRNGs across numerous streams and tests, providing a comprehensive comparison against their claimed qualities and documenting their failures.

Findings

Highest success rate of 72% in tests

Almost all generators failed some tests

Results are reproducible and available in a GitHub repository

Abstract

Pseudo-random number generators (PRNGs) are widely used in modern computing and are expected to exhibit excellent statistical performance and repeatability. This study evaluates and compares modern PRNGs used in high performance computing and artificial intelligence. Our selections comes from different families, including Xoshiro, Philox, PCG, and MRG32k3a. We systematically assess the quality of these generators; instead of testing a single stream for each generator, we test more than 10 3 streams with the BigCrush battery form the TestU01 library. The results, involving more than 4.5 years of cumulative computing time, are analyzed against the claims made by the generators' creators. The highest success rate is 72%, and all tests have been failed by almost every generator, the failed tests are documented. To ensure fairness, all tests are conducted under consistent conditions and are designed to closely simulate real-world usage. The results of each test are available, usable and reproducible with a git repository.