Bisimulations and Logical Characterizations on Continuous-time Markov Decision Processes

TL;DR

This paper investigates strong and weak bisimulation relations for continuous-time Markov decision processes (CTMDPs) and their logical characterizations using continuous-time stochastic logic (CSL), providing new insights into their equivalences.

Contribution

It introduces strong and weak bisimulations for CTMDPs and proves their soundness and completeness with respect to CSL and its sub-logic, extending logical characterizations.

Findings

Strong and weak bisimulations are sound and complete for a subclass of CTMDPs.

Full logical characterizations are achieved for an extended CSL and its sub-logic.

The relations differ in their fineness and applicability to arbitrary CTMDPs.

Abstract

In this paper we study strong and weak bisimulation equivalences for continuous-time Markov decision processes (CTMDPs) and the logical characterizations of these relations with respect to the continuous-time stochastic logic (CSL). For strong bisimulation, it is well known that it is strictly finer than CSL equivalence. In this paper we propose strong and weak bisimulations for CTMDPs and show that for a subclass of CTMDPs, strong and weak bisimulations are both sound and complete with respect to the equivalences induced by CSL and the sub-logic of CSL without next operator respectively. We then consider a standard extension of CSL, and show that it and its sub-logic without X can be fully characterized by strong and weak bisimulations respectively over arbitrary CTMDPs.