Computing Expected Hitting Times for Imprecise Markov Chains

TL;DR

This paper introduces a new algorithm for efficiently computing tight bounds on expected hitting times in imprecise Markov chains with partially specified transition probabilities, improving upon existing methods.

Contribution

The paper presents a novel algorithm for solving a non-linear system to find lower and upper bounds on expected hitting times in imprecise Markov chains, with proven correctness and finite convergence.

Findings

Algorithm computes tight bounds on expected hitting times.

Proven convergence in finite steps under mild conditions.

Outperforms previous methods in certain scenarios.

Abstract

We present a novel algorithm to solve a non-linear system of equations, whose solution can be interpreted as a tight lower bound on the vector of expected hitting times of a Markov chain whose transition probabilities are only partially specified. We also briefly sketch how this method can be modified to solve a conjugate system of equations that gives rise to the corresponding upper bound. We prove the correctness of our method, and show that it converges to the correct solution in a finite number of steps under mild conditions on the system. We compare the runtime complexity of our method to a previously published method from the literature, and identify conditions under which our novel method is more efficient.