Planning Multiple Epidemic Interventions with Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning framework to optimize complex, continuous epidemic intervention plans, effectively balancing health and economic outcomes, and demonstrating superior performance over traditional methods.

Contribution

It formulates epidemic intervention planning as a Markov decision process with continuous actions and states, applying actor-critic RL algorithms to find optimal policies.

Findings

RL algorithms outperform hand-crafted baselines

The approach effectively handles complex, continuous intervention spaces

Proves the viability of computational tools for policy support

Abstract

Combating an epidemic entails finding a plan that describes when and how to apply different interventions, such as mask-wearing mandates, vaccinations, school or workplace closures. An optimal plan will curb an epidemic with minimal loss of life, disease burden, and economic cost. Finding an optimal plan is an intractable computational problem in realistic settings. Policy-makers, however, would greatly benefit from tools that can efficiently search for plans that minimize disease and economic costs especially when considering multiple possible interventions over a continuous and complex action space given a continuous and equally complex state space. We formulate this problem as a Markov decision process. Our formulation is unique in its ability to represent multiple continuous interventions over any disease model defined by ordinary differential equations. We illustrate how to effectively apply state-of-the-art actor-critic reinforcement learning algorithms (PPO and SAC) to search for plans that minimize overall costs. We empirically evaluate the learning performance of these algorithms and compare their performance to hand-crafted baselines that mimic plans constructed by policy-makers. Our method outperforms baselines. Our work confirms the viability of a computational approach to support policy-makers