Deep Reinforcement Learning of Transition States

TL;DR

This paper introduces a novel machine learning method combining reinforcement learning and molecular dynamics to automatically identify and analyze chemical reaction transition states and mechanisms with minimal bias.

Contribution

The authors develop RL$^ ext{ extdagger extdagger}$, a reinforcement learning framework that formulates transition state discovery as a game, enabling automatic, unbiased analysis of reaction mechanisms.

Findings

RL$^ ext{ extdagger extdagger}$ effectively uncovers reaction mechanisms.

The method allows efficient sampling of transition paths.

It provides interpretable insights into reaction dynamics.

Abstract

Combining reinforcement learning (RL) and molecular dynamics (MD) simulations, we propose a machine-learning approach (RL$^\ddag$) to automatically unravel chemical reaction mechanisms. In RL$^\ddag$, locating the transition state of a chemical reaction is formulated as a game, where a virtual player is trained to shoot simulation trajectories connecting the reactant and product. The player utilizes two functions, one for value estimation and the other for policy making, to iteratively improve the chance of winning this game. We can directly interpret the reaction mechanism according to the value function. Meanwhile, the policy function enables efficient sampling of the transition paths, which can be further used to analyze the reaction dynamics and kinetics. Through multiple experiments, we show that RL{\ddag} can be trained tabula rasa hence allows us to reveal chemical reaction mechanisms with minimal subjective biases.