Evolutionary reinforcement learning of dynamical large deviations

TL;DR

This paper introduces an evolutionary reinforcement learning method to compute the likelihood of large deviations in dynamical systems, bridging physics and machine learning techniques.

Contribution

It presents a novel approach combining evolutionary algorithms and reinforcement learning to calculate large deviation rate functions for complex models.

Findings

Effective in small state space models by acting directly on rates.

Scalable to large state spaces using neural network parameterization.

Demonstrates integration of physics problems with machine learning frameworks.

Abstract

We show how to calculate the likelihood of dynamical large deviations using evolutionary reinforcement learning. An agent, a stochastic model, propagates a continuous-time Monte Carlo trajectory and receives a reward conditioned upon the values of certain path-extensive quantities. Evolution produces progressively fitter agents, eventually allowing the calculation of a piece of a large-deviation rate function for a particular model and path-extensive quantity. For models with small state spaces the evolutionary process acts directly on rates, and for models with large state spaces the process acts on the weights of a neural network that parameterizes the model's rates. This approach shows how path-extensive physics problems can be considered within a framework widely used in machine learning.