Strategy Complexity of B\"uchi and Transience Objectives in Concurrent Stochastic Games

TL;DR

This paper investigates the complexity of strategies in stochastic B"uchi and Transience games on countable graphs, showing that simple strategies with minimal memory are sufficient for near-optimal play.

Contribution

It introduces tight bounds on the memory needed for $ ext{epsilon}$-optimal strategies in B"uchi and Transience objectives, including new results for finite and infinite graphs.

Findings

Existence of $ ext{epsilon}$-optimal strategies with 1 bit of public memory for B"uchi games.

Memoryless strategies suffice for Transience objectives.

Upper bounds are tight even for finite graphs.

Abstract

We study 2-player zero-sum concurrent (i.e., simultaneous move) stochastic B\"uchi games and Transience games on countable graphs. Two players, Max and Min, seek respectively to maximize and minimize the probability of satisfying the game objective. The B\"uchi objective is to visit a given set of target states infinitely often. This can be seen as a special case of maximizing the expected $\limsup$ of the daily rewards, where all daily rewards are in $\{0,1\}$. The Transience objective is to visit no state infinitely often, i.e., every finite subset of the states is eventually left forever. Transience can only be met in infinite game graphs. We show that in B\"uchi games there always exist $\varepsilon$-optimal Max strategies that use just a step counter (discrete clock) plus 1 bit of public memory. This upper bound holds for all countable graphs, but it is a new result even for the special case of finite graphs. The upper bound is tight in the sense that Max strategies that use just a step counter, or just finite memory, are not sufficient even on finite game graphs. This upper bound is a consequence of a slightly stronger new result: $\varepsilon$-optimal Max strategies for the combined B\"uchi and Transience objective require just 1 bit of public memory (but cannot be memoryless). Our proof techniques also yield a closely related result, that $\varepsilon$-optimal Max strategies for the Transience objective alone can be chosen as memoryless.