Towards neural reinforcement learning for large deviations in nonequilibrium systems with memory

TL;DR

This paper develops a neural reinforcement learning approach for analyzing large deviations in non-Markovian systems, especially semi-Markov processes, by extending actor-critic methods with memory processing neural networks.

Contribution

It introduces a novel neural reinforcement learning framework that incorporates memory variables for studying fluctuations in non-Markovian systems.

Findings

Successfully applied to current fluctuations in semi-Markov models

Demonstrates effectiveness in handling nonexponential waiting times

Extends existing actor-critic methods with memory neural networks

Abstract

We introduce a reinforcement learning method for a class of non-Markov systems; our approach extends the actor-critic framework given by Rose et al. [New J. Phys. 23 013013 (2021)] for obtaining scaled cumulant generating functions characterizing the fluctuations. The actor-critic is implemented using neural networks; a particular innovation in our method is the use of an additional neural policy for processing memory variables. We demonstrate results for current fluctuations in various memory-dependent models with special focus on semi-Markov systems where the dynamics is controlled by nonexponential interevent waiting time distributions.