Definable zero-sum stochastic games

TL;DR

This paper studies zero-sum stochastic games with data definable in o-minimal structures, proving the definability of the Shapley operator and the existence of a uniform value under certain conditions, with applications to control theory.

Contribution

It establishes the definability of the Shapley operator for a broad class of definable stochastic games and proves the existence of uniform values, including convergence rates for polynomially bounded cases.

Findings

Shapley operator is definable in the same structure for separable definable games.

Separable definable games have a uniform value.

Applications include polynomial transitions, finite actions, perfect information, and switching controls.

Abstract

Definable zero-sum stochastic games involve a finite number of states and action sets, reward and transition functions that are definable in an o-minimal structure. Prominent examples of such games are finite, semi-algebraic or globally subanalytic stochastic games. We prove that the Shapley operator of any definable stochastic game with separable transition and reward functions is definable in the same structure. Definability in the same structure does not hold systematically: we provide a counterexample of a stochastic game with semi-algebraic data yielding a non semi-algebraic but globally subanalytic Shapley operator. Our definability results on Shapley operators are used to prove that any separable definable game has a uniform value; in the case of polynomially bounded structures we also provide convergence rates. Using an approximation procedure, we actually establish that general zero-sum games with separable definable transition functions have a uniform value. These results highlight the key role played by the tame structure of transition functions. As particular cases of our main results, we obtain that stochastic games with polynomial transitions, definable games with finite actions on one side, definable games with perfect information or switching controls have a uniform value. Applications to nonlinear maps arising in risk sensitive control and Perron-Frobenius theory are also given