The compositional construction of Markov processes II

TL;DR

This paper extends the algebraic framework for Markov automata by adding sequential operations, enabling the compositional modeling of hierarchical systems and systems with evolving geometry, with implications for analyzing system behavior over time.

Contribution

It introduces sequential operations into the algebra of Markov automata, allowing for more complex and hierarchical system descriptions.

Findings

Enables modeling of hierarchical systems.

Supports systems with evolving geometry.

Provides a foundation for analyzing system dynamics.

Abstract

In an earlier paper we introduced a notion of Markov automaton, together with parallel operations which permit the compositional description of Markov processes. We illustrated by showing how to describe a system of n dining philosophers, and we observed that Perron-Frobenius theory yields a proof that the probability of reaching deadlock tends to one as the number of steps goes to infinity. In this paper we add sequential operations to the algebra (and the necessary structure to support them). The extra operations permit the description of hierarchical systems, and ones with evolving geometry.