Strategy Complexity of Reachability in Countable Stochastic 2-Player Games

TL;DR

This paper analyzes the memory requirements for optimal strategies in countably infinite stochastic 2-player reachability games, revealing that infinite memory is often necessary and that simple strategies may not suffice.

Contribution

It provides a complete characterization of the memory complexity of optimal strategies depending on action set sizes and uniformity constraints, including new lower bounds.

Findings

Infinite memory needed for optimal strategies with infinite action sets.

Finite-memory strategies are ineffective in certain infinite branching games.

Existence of simple one-bit memory strategies in finite action set games.

Abstract

We study countably infinite stochastic 2-player games with reachability objectives. Our results provide a complete picture of the memory requirements of $\varepsilon$-optimal (resp. optimal) strategies. These results depend on the size of the players' action sets and on whether one requires strategies that are uniform (i.e., independent of the start state). Our main result is that $\varepsilon$-optimal (resp. optimal) Maximizer strategies require infinite memory if Minimizer is allowed infinite action sets. This lower bound holds even under very strong restrictions. Even in the special case of infinitely branching turn-based reachability games, even if all states allow an almost surely winning Maximizer strategy, strategies with a step counter plus finite private memory are still useless. Regarding uniformity, we show that for Maximizer there need not exist positional (i.e., memoryless) uniformly $\varepsilon$-optimal strategies even in the special case of finite action sets or in finitely branching turn-based games. On the other hand, in games with finite action sets, there always exists a uniformly $\varepsilon$-optimal Maximizer strategy that uses just one bit of public memory.