Conversion of Mersenne Twister to double-precision floating-point numbers

TL;DR

This paper analyzes how converting 32-bit Mersenne Twister outputs into 53-bit double-precision floating-point numbers affects its statistical properties and reveals certain linear relations and test failures.

Contribution

It investigates the impact of concatenating 32-bit outputs on the statistical quality of MT19937 and identifies specific conditions where it fails standard randomness tests.

Findings

MT19937 with certain lag sets fails statistical tests

Concatenation can degrade equidistribution properties

Hidden linear relations among bits are identified

Abstract

The 32-bit Mersenne Twister generator MT19937 is a widely used random number generator. To generate numbers with more than 32 bits in bit length, and particularly when converting into 53-bit double-precision floating-point numbers in $[0,1)$ in the IEEE 754 format, the typical implementation concatenates two successive 32-bit integers and divides them by a power of $2$. In this case, the 32-bit MT19937 is optimized in terms of its equidistribution properties (the so-called dimension of equidistribution with $v$-bit accuracy) under the assumption that one will mainly be using 32-bit output values, and hence the concatenation sometimes degrades the dimension of equidistribution compared with the simple use of 32-bit outputs. In this paper, we analyze such phenomena by investigating hidden $\mathbb{F}_2$-linear relations among the bits of high-dimensional outputs. Accordingly, we report that MT19937 with a specific lag set fails several statistical tests, such as the overlapping collision test, matrix rank test, and Hamming independence test.