Reinforcement learning of rare diffusive dynamics

TL;DR

This paper introduces a reinforcement learning approach to efficiently sample and analyze rare molecular trajectories, enabling better estimation of their likelihoods and underlying forces in complex systems.

Contribution

It develops a novel reinforcement learning method that optimizes forces to probe rare events and fluctuations in molecular dynamics, improving accuracy and efficiency over existing techniques.

Findings

Successfully estimates likelihood of rare events in model systems.

Provides an optimized force that transforms rare trajectories into typical ones.

Enhances convergence with low variance gradient techniques.

Abstract

We present a method to probe rare molecular dynamics trajectories directly using reinforcement learning. We consider trajectories that are conditioned to transition between regions of configuration space in finite time, like those relevant in the study of reactive events, as well as trajectories exhibiting rare fluctuations of time-integrated quantities in the long time limit, like those relevant in the calculation of large deviation functions. In both cases, reinforcement learning techniques are used to optimize an added force that minimizes the Kullback-Leibler divergence between the conditioned trajectory ensemble and a driven one. Under the optimized added force, the system evolves the rare fluctuation as a typical one, affording a variational estimate of its likelihood in the original trajectory ensemble. Low variance gradients employing value functions are proposed to increase the convergence of the optimal force. The method we develop employing these gradients leads to efficient and accurate estimates of both the optimal force and the likelihood of the rare event for a variety of model systems.