Maximal Cost-Bounded Reachability Probability on Continuous-Time Markov Decision Processes

TL;DR

This paper studies the maximum probability of reaching certain states within cost bounds in continuous-time Markov decision processes, providing integral characterizations, optimal schedulers, and approximation algorithms.

Contribution

It introduces an integral characterization of the probability function, proves the existence of optimal deterministic schedulers, and offers a numerical approximation method.

Findings

Probability function is the least fixed point of integral equations.

Optimal schedulers are measurable and deterministic.

Provides a numerical algorithm for approximation.

Abstract

In this paper, we consider multi-dimensional maximal cost-bounded reachability probability over continuous-time Markov decision processes (CTMDPs). Our major contributions are as follows. Firstly, we derive an integral characterization which states that the maximal cost-bounded reachability probability function is the least fixed point of a system of integral equations. Secondly, we prove that the maximal cost-bounded reachability probability can be attained by a measurable deterministic cost-positional scheduler. Thirdly, we provide a numerical approximation algorithm for maximal cost-bounded reachability probability. We present these results under the setting of both early and late schedulers.