Control of Probabilistic Systems under Dynamic, Partially Known Environments with Temporal Logic Specifications

TL;DR

This paper develops methods for synthesizing control policies for probabilistic systems operating in partially known environments, modeled by Markov chains, with the goal of satisfying temporal logic specifications under uncertainty.

Contribution

It introduces a framework for control synthesis in environments with unknown modes, considering both expected and worst-case satisfaction probabilities.

Findings

Proposes control strategies for probabilistic systems in uncertain environments.

Addresses both expected and worst-case satisfaction probabilities.

Provides theoretical guarantees for control policy effectiveness.

Abstract

We consider the synthesis of control policies for probabilistic systems, modeled by Markov decision processes, operating in partially known environments with temporal logic specifications. The environment is modeled by a set of Markov chains. Each Markov chain describes the behavior of the environment in each mode. The mode of the environment, however, is not known to the system. Two control objectives are considered: maximizing the expected probability and maximizing the worst-case probability that the system satisfies a given specification.