VacSIM: Learning Effective Strategies for COVID-19 Vaccine Distribution using Reinforcement Learning

TL;DR

VacSIM employs deep reinforcement learning combined with contextual bandits to optimize COVID-19 vaccine distribution, significantly reducing infections in Indian states and offering a scalable, open-source platform for global application.

Contribution

This paper introduces a novel reinforcement learning-based framework, VacSIM, for optimizing vaccine distribution with real-time adaptability and extensive evaluation strategies.

Findings

Up to 9039 infections potentially prevented in 45 days

Significant improvement over naive proportional distribution

Framework extensible to all Indian states and globally

Abstract

A COVID-19 vaccine is our best bet for mitigating the ongoing onslaught of the pandemic. However, vaccine is also expected to be a limited resource. An optimal allocation strategy, especially in countries with access inequities and temporal separation of hot-spots, might be an effective way of halting the disease spread. We approach this problem by proposing a novel pipeline VacSIM that dovetails Deep Reinforcement Learning models into a Contextual Bandits approach for optimizing the distribution of COVID-19 vaccine. Whereas the Reinforcement Learning models suggest better actions and rewards, Contextual Bandits allow online modifications that may need to be implemented on a day-to-day basis in the real world scenario. We evaluate this framework against a naive allocation approach of distributing vaccine proportional to the incidence of COVID-19 cases in five different States across India (Assam, Delhi, Jharkhand, Maharashtra and Nagaland) and demonstrate up to 9039 potential infections prevented and a significant increase in the efficacy of limiting the spread over a period of 45 days through the VacSIM approach. Our models and the platform are extensible to all states of India and potentially across the globe. We also propose novel evaluation strategies including standard compartmental model-based projections and a causality-preserving evaluation of our model. Since all models carry assumptions that may need to be tested in various contexts, we open source our model VacSIM and contribute a new reinforcement learning environment compatible with OpenAI gym to make it extensible for real-world applications across the globe. (http://vacsim.tavlab.iiitd.edu.in:8000/).