Controlling dynamics of stochastic systems with deep reinforcement learning

TL;DR

This paper introduces a simulation algorithm that uses deep reinforcement learning and neural networks to control the dynamics of stochastic systems, demonstrated on particle coalescence and exclusion processes.

Contribution

It bridges control theory and deep reinforcement learning by proposing a novel simulation scheme for controlling stochastic system dynamics with trained neural networks.

Findings

Effective control of stochastic processes demonstrated

Neural network controllers successfully guide system dynamics

Workflow applicable to complex stochastic systems

Abstract

A properly designed controller can help improve the quality of experimental measurements or force a dynamical system to follow a completely new time-evolution path. Recent developments in deep reinforcement learning have made steep advances toward designing effective control schemes for fairly complex systems. However, a general simulation scheme that employs deep reinforcement learning for exerting control in stochastic systems is yet to be established. In this paper, we attempt to further bridge a gap between control theory and deep reinforcement learning by proposing a simulation algorithm that allows achieving control of the dynamics of stochastic systems through the use of trained artificial neural networks. Specifically, we use agent-based simulations where the neural network plays the role of the controller that drives local state-to-state transitions. We demonstrate the workflow and the effectiveness of the proposed control methods by considering the following two stochastic processes: particle coalescence on a lattice and a totally asymmetric exclusion process.