Quantitative model-checking of controlled discrete-time Markov processes

TL;DR

This paper develops methods for optimizing event probabilities in controlled discrete-time Markov processes, reducing complex properties to reachability problems and providing a detailed analysis of these problems with a case study.

Contribution

It introduces a reduction of complex property optimization to reachability problems and offers a comprehensive analysis of reachability in controlled Markov processes.

Findings

Optimization reduces to reachability and repeated reachability problems.

Provides a detailed study of reachability in controlled Markov processes.

Includes a case study demonstrating the techniques.

Abstract

This paper focuses on optimizing probabilities of events of interest defined over general controlled discrete-time Markov processes. It is shown that the optimization over a wide class of $\omega$-regular properties can be reduced to the solution of one of two fundamental problems: reachability and repeated reachability. We provide a comprehensive study of the former problem and an initial characterisation of the (much more involved) latter problem. A case study elucidates concepts and techniques.