Probabilistic Systems with LimSup and LimInf Objectives

TL;DR

This paper presents polynomial-time algorithms for computing the values of Markov decision processes with limsup and liminf objectives, and extends these results to solve related stochastic games within NP ∩ coNP.

Contribution

It introduces efficient algorithms for MDPs with limsup/liminf objectives and demonstrates their application to solving turn-based stochastic games in NP ∩ coNP.

Findings

Polynomial-time algorithms for MDPs with limsup/liminf objectives

Turn-based stochastic games with these objectives are in NP ∩ coNP

Enhanced understanding of stochastic decision processes with infinite-horizon objectives

Abstract

We give polynomial-time algorithms for computing the values of Markov decision processes (MDPs) with limsup and liminf objectives. A real-valued reward is assigned to each state, and the value of an infinite path in the MDP is the limsup (resp. liminf) of all rewards along the path. The value of an MDP is the maximal expected value of an infinite path that can be achieved by resolving the decisions of the MDP. Using our result on MDPs, we show that turn-based stochastic games with limsup and liminf objectives can be solved in NP \cap coNP.