Hitting Times and Probabilities for Imprecise Markov Chains

TL;DR

This paper unifies the analysis of expected hitting times and probabilities across different types of imprecise Markov chains, providing a generalized characterization that extends classical results for precise chains.

Contribution

It demonstrates that various definitions of imprecise Markov chains share the same hitting time and probability bounds, and offers a generalized characterization method.

Findings

All types of imprecise Markov chains have identical lower and upper expected hitting times.

Hitting probabilities are consistent across the three types of imprecise Markov chains.

Provides a generalized characterization of hitting times and probabilities for these chains.

Abstract

We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as sets of homogeneous Markov chains, as sets of more general stochastic processes, and as game-theoretic probability models. Our first contribution is that all these different types of imprecise Markov chains have the same lower and upper expected hitting times, and similarly the hitting probabilities are the same for these three types. Moreover, we provide a characterisation of these quantities that directly generalises a similar characterisation for precise, homogeneous Markov chains.