Strategy Complexity of Concurrent Stochastic Games with Safety and Reachability Objectives

TL;DR

This paper analyzes the complexity of stationary strategies in finite-state concurrent stochastic games with reachability and safety objectives, revealing doubly exponential patience bounds in certain cases and exponential bounds in others.

Contribution

It provides the first tight bounds on the patience required for optimal and epsilon-Nash strategies in these classes of stochastic games.

Findings

Doubly exponential patience bounds for two-player zero-sum games with reachability and safety objectives.

Exponential patience bounds are necessary even with a single nonabsorbing state.

Doubly-exponential patience may be needed for epsilon-Nash equilibria when at least one player has a reachability objective.

Abstract

We consider finite-state concurrent stochastic games, played by k>=2 players for an infinite number of rounds, where in every round, each player simultaneously and independently of the other players chooses an action, whereafter the successor state is determined by a probability distribution given by the current state and the chosen actions. We consider reachability objectives that given a target set of states require that some state in the target is visited, and the dual safety objectives that given a target set require that only states in the target set are visited. We are interested in the complexity of stationary strategies measured by their patience, which is defined as the inverse of the smallest nonzero probability employed. Our main results are as follows: We show that in two-player zero-sum concurrent stochastic games (with reachability objective for one player and the complementary safety objective for the other player): (i) the optimal bound on the patience of optimal and epsilon-optimal strategies, for both players is doubly exponential; and (ii) even in games with a single nonabsorbing state exponential (in the number of actions) patience is necessary. In general we study the class of non-zero-sum games admitting stationary epsilon-Nash equilibria. We show that if there is at least one player with reachability objective, then doubly-exponential patience may be needed for epsilon-Nash equilibrium strategies, whereas in contrast if all players have safety objectives, the optimal bound on patience for epsilon-Nash equilibrium strategies is only exponential.