Lower and Upper Expected Hitting Times for Weighted Imprecise Markov Chains

TL;DR

This paper extends the concept of hitting times to weighted imprecise Markov chains, providing a framework for calculating bounds and adapting existing algorithms for these complex models.

Contribution

It introduces weighted expected hitting times for WIMCs, characterizes them via fixed-point equations, and adapts existing methods for their numerical computation.

Findings

Defined lower and upper weighted expected hitting times.

Characterized these times as solutions to nonlinear fixed-point equations.

Demonstrated transformation to unweighted problems for algorithm reuse.

Abstract

In this paper, we extend hitting times for imprecise Markov chains to the framework of weighted imprecise Markov chains (WIMCs), in which each transition is associated with a strictly positive weight encoded by a matrix $W$. Given a convex set $\mathcal{T}$ of admissible transition matrices, we define lower and upper expected hitting times for WIMCs as the infimum and supremum of the (weighted) expected hitting times over $\mathcal{T}$, and we characterise these quantities as the unique solutions of nonlinear fixed-point equations. We show that any weighted hitting time problem can be transformed into an unweighted hitting time problem on an augmented state space, enabling the reuse of existing IMC theory and algorithms. In particular, we are able to adapt known iterative methods for the numerical computation of expected hitting times for WIMCs.