Entropy of Pseudo Random Number Generators

TL;DR

This paper investigates the entropy properties of pseudo random number generators, revealing that low conditional entropy in their production rules leads to failures in cluster Monte Carlo simulations, and proposes entropy as a fundamental quality measure.

Contribution

It introduces the concept of entropy of the generator's rule as a fundamental measure of randomness quality, explaining failures in simulations.

Findings

Low entropy of the generator's rule causes simulation failures

Conditional entropy is independent of sequence lag or period

Entropy provides a deeper quality measure than empirical tests

Abstract

Since the work of Ferrenberg et al.[PRL 69, (1992)] some pseudo random number generators are known to yield wrong results in cluster Monte Carlo simulations. In this contribution the fundamental mechanism behind this failure is discussed. Almost all random number generators calculate a new pseudo random number $x_i$ from preceding values, $x_i = f(x_{i-1}, x_{i-2},..., x_{i-q})$. Failure of these generators in cluster Monte Carlo simulations and related experiments can be attributed to the low entropy of the production rule $f()$ conditioned on the statistics of the input values $x_{i-1},...,x_{i-q}$. Being a measure only of the arithmetic operations in the generator rule, the conditional entropy is independent of the lag in the recurrence or the period of the sequence. In that sense it measures a more profound quality of a random number generator than empirical tests with their limited horizon.