Pseudo Random Number Generation through Reinforcement Learning and Recurrent Neural Networks

TL;DR

This paper introduces a novel reinforcement learning method using LSTM networks to generate pseudo-random numbers, improving upon previous approaches by modeling the process as a partially observable Markov decision process.

Contribution

It presents a new RL-based framework employing LSTM to generate PRNGs from scratch, capturing temporal dependencies and partial observability for better randomness quality.

Findings

LSTM-based RL significantly outperforms fully observable models.

Modeling partial observability improves PRNG quality.

The approach effectively learns to generate sequences with desired statistical properties.

Abstract

A Pseudo-Random Number Generator (PRNG) is any algorithm generating a sequence of numbers approximating properties of random numbers. These numbers are widely employed in mid-level cryptography and in software applications. Test suites are used to evaluate PRNGs quality by checking statistical properties of the generated sequences. These sequences are commonly represented bit by bit. This paper proposes a Reinforcement Learning (RL) approach to the task of generating PRNGs from scratch by learning a policy to solve a partially observable Markov Decision Process (MDP), where the full state is the period of the generated sequence and the observation at each time step is the last sequence of bits appended to such state. We use a Long-Short Term Memory (LSTM) architecture to model the temporal relationship between observations at different time steps, by tasking the LSTM memory with the extraction of significant features of the hidden portion of the MDP's states. We show that modeling a PRNG with a partially observable MDP and a LSTM architecture largely improves the results of the fully observable feedforward RL approach introduced in previous work.