Optimal control of stochastic reaction networks with entropic control cost and emergence of mode-switching strategies

TL;DR

This paper develops a novel optimal control framework for stochastic biological populations using f-divergence-based costs, simplifying complex equations and revealing mode-switching strategies in control.

Contribution

It introduces a new control formulation for nonlinear, discrete stochastic systems using f-divergence, enabling efficient solutions and uncovering mode-switching behaviors.

Findings

Efficient control solutions for stochastic population models.

Observation of mode-switching phenomena in control strategies.

Simplification of complex control equations using KL divergence.

Abstract

Controlling the stochastic dynamics of biological populations is a challenge that arises across various biological contexts. However, these dynamics are inherently nonlinear and involve a discrete state space, i.e., the number of molecules, cells, or organisms. Additionally, the possibility of extinction has a significant impact on both dynamics and control strategies, particularly when the population size is small. These factors hamper the direct application of conventional control theories to biological systems. To address these challenges, we formulate the optimal control problem for stochastic population dynamics by utilizing control cost functions based on the f-divergence, which naturally accounts for population-specific factors. If Kullback-Leibler (KL) divergence is adopted for the cost function, the complex nonlinear Hamilton-Jacobi-Bellman equation is simplified into a linear form, facilitating efficient computation of optimal solutions. We demonstrate the effectiveness of our approach by applying it to the control of interacting random walkers, Moran processes, and SIR models, and observe the mode-switching phenomena in the control strategies. Our approach provides new opportunities for applying control theory to a wide range of biological problems.