MM Algorithms to Estimate Parameters in Continuous-time Markov Chains

TL;DR

This paper introduces MM algorithms for estimating parameters in continuous-time Markov chains modeled as parametric CTMCs, enabling parameter inference from partially-observable data in systems like COVID-19 spread analysis.

Contribution

It develops iterative maximum likelihood estimation algorithms for parametric CTMCs, covering scenarios with different levels of observability, based on the MM algorithm framework.

Findings

Algorithms successfully estimate parameters from partial observations.

Application to COVID-19 spread demonstrates practical utility.

Method outperforms existing approaches in accuracy and efficiency.

Abstract

Continuous-time Markov chains (CTMCs) are popular modeling formalism that constitutes the underlying semantics for real-time probabilistic systems such as queuing networks, stochastic process algebras, and calculi for systems biology. Prism and Storm are popular model checking tools that provide a number of powerful analysis techniques for CTMCs. These tools accept models expressed as the parallel composition of a number of modules interacting with each other. The outcome of the analysis is strongly dependent on the parameter values used in the model which govern the timing and probability of events of the resulting CTMC. However, for some applications, parameter values have to be empirically estimated from partially-observable executions. In this work, we address the problem of estimating parameter values of CTMCs expressed as Prism models from a number of partially-observable executions. We introduce the class parametric CTMCs -- CTMCs where transition rates are polynomial functions over a set of parameters -- as an abstraction of CTMCs covering a large class of Prism models. Then, building on a theory of algorithms known by the initials MM, for minorization-maximization, we present iterative maximum likelihood estimation algorithms for parametric CTMCs covering two learning scenarios: when both state-labels and dwell times are observable, or just state-labels are. We conclude by illustrating the use of our technique in a simple but non-trivial case study: the analysis of the spread of COVID-19 in presence of lockdown countermeasures.