Determinacy in Stochastic Games with Unbounded Payoff Functions

TL;DR

This paper proves determinacy for infinite-state stochastic games with unbounded rewards, showing that such games are determined under various strategies, and explores conditions for optimal strategies.

Contribution

It establishes determinacy results for unbounded reward stochastic games, including for deterministic and history-dependent strategies, and analyzes subclasses with guaranteed determinacy.

Findings

Games are determined for unrestricted and deterministic strategies.

Games are not generally determined for memoryless strategies.

Finitely-branching subclasses are determined for all strategy types.

Abstract

We consider infinite-state turn-based stochastic games of two players, Box and Diamond, who aim at maximizing and minimizing the expected total reward accumulated along a run, respectively. Since the total accumulated reward is unbounded, the determinacy of such games cannot be deduced directly from Martin's determinacy result for Blackwell games. Nevertheless, we show that these games are determined both for unrestricted (i.e., history-dependent and randomized) strategies and deterministic strategies, and the equilibrium value is the same. Further, we show that these games are generally not determined for memoryless strategies. Then, we consider a subclass of Diamond-finitely-branching games and show that they are determined for all of the considered strategy types, where the equilibrium value is always the same. We also examine the existence and type of (epsilon-)optimal strategies for both players.