Imprecise Continuous-Time Markov Chains

TL;DR

This paper introduces imprecise continuous-time Markov chains (ICTMCs), a robust extension of traditional models that relax assumptions like exact parameters and time-homogeneity, enabling computation of lower expectations with polynomial complexity.

Contribution

It formalizes ICTMCs, characterizes their properties, and develops polynomial-time algorithms for lower expectation computations under relaxed assumptions.

Findings

ICTMCs generalize classical CTMCs by relaxing key assumptions.

Lower expectations can be computed efficiently using the proposed algorithms.

The formalism provides a foundation for robust stochastic modeling in uncertain environments.

Abstract

Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models computationally tractable, they rely on a number of assumptions that may not be realistic for the domain of application; in particular, the ability to provide exact numerical parameter assessments, and the applicability of time-homogeneity and the eponymous Markov property. In this work, we extend these models to imprecise continuous-time Markov chains (ICTMC's), which are a robust generalisation that relaxes these assumptions while remaining computationally tractable. More technically, an ICTMC is a set of "precise" continuous-time finite-state stochastic processes, and rather than computing expected values of functions, we seek to compute lower expectations, which are tight lower bounds on the expectations that correspond to such a set of "precise" models. Note that, in contrast to e.g. Bayesian methods, all the elements of such a set are treated on equal grounds; we do not consider a distribution over this set. The first part of this paper develops a formalism for describing continuous-time finite-state stochastic processes that does not require the aforementioned simplifying assumptions. Next, this formalism is used to characterise ICTMC's and to investigate their properties. The concept of lower expectation is then given an alternative operator-theoretic characterisation, by means of a lower transition operator, and the properties of this operator are investigated as well. Finally, we use this lower transition operator to derive tractable algorithms (with polynomial runtime complexity w.r.t. the maximum numerical error) for computing the lower expectation of functions that depend on the state at any finite number of time points.