Stationary and Non-Stationary Transition Probabilities in Decision Making: Modeling COVID-19 Dynamics

TL;DR

This paper compares stationary and non-stationary Markov models to better understand COVID-19 dynamics, emphasizing the importance of time-dependent transitions for accurate epidemic modeling and intervention planning.

Contribution

It introduces a comprehensive framework for modeling COVID-19 with both stationary and non-stationary transition probabilities within an MDP, highlighting the significance of non-stationary approaches.

Findings

Non-stationary models better capture epidemic dynamics.

Time-dependent transitions improve intervention strategies.

Simulation over a year demonstrates dynamic variations.

Abstract

This study introduces a comparative modeling framework using stationary and non-stationary transition probabilities within a Markov Decision Process (MDP) to assess COVID-19 disease dynamics. Stationary transition probabilities assume constant transition rates, while non-stationary transitions reflect time-dependent behaviors including policy interventions or behavioral changes. We develop a comprehensive compartmental model with transitions based on binomial and multinomial processes. Mathematical models for both stationary and non-stationary transition frameworks are developed and simulated over a 365-day period to emphasize dynamic variations in epidemic outcomes. Our findings highlight the significance of non-stationary modeling in accurately representing the dynamic characteristics of pandemic situations and provide recommendations for optimizing public health interventions under uncertainty. This comparative analysis offers useful information for epidemiological modeling and decision-making in dynamic risk environments.