Development of COVID-19 Booster Vaccine Policy by Microsimulation and Q-learning

TL;DR

This paper introduces a novel framework combining microsimulation with Q-learning to develop ethical, effective COVID-19 booster vaccination policies without real-world trial risks.

Contribution

It presents a new approach integrating RNN-based microsimulation with tabular Q-learning for vaccine policy optimization, addressing ethical and exploration challenges.

Findings

Q-learning policy outperforms current vaccination strategies

The RNN simulator captures realistic infection dynamics

The approach ensures ethical policy development without real-world testing

Abstract

The COVID-19 pandemic highlighted the urgent need for effective vaccine policies, but traditional clinical trials often lack sufficient data to capture the diverse population characteristics necessary for comprehensive public health strategies. Ethical concerns around randomized trials during a pandemic further complicate policy development for public health. Reinforcement Learning (RL) offers a promising alternative for vaccine policy development. However, direct online RL exploration in real-world scenarios can result in suboptimal and potentially harmful decisions. This study proposes a novel framework combining tabular Q-learning with microsimulation (i.e., a Recurrent Neural Network (RNN) environment simulator) to address these challenges in public health vaccine policymaking, which enables effective vaccine policy learning without real-world interaction, addressing both ethical and exploration challenges. The RNN environment simulator captures temporal associations between infection and patient characteristics, generating realistic simulation data. Our tabular Q-learning model produces an interpretable policy table that balances the risks of severe infection against vaccination side effects. Applied to COVID-19 booster policies, the learned Q-learning-based policy outperforms current practices, offering a path toward more effective vaccination strategies.