Commutative Stochastic Games

TL;DR

This paper investigates the convergence and existence of uniform values in commutative stochastic games with general states and finite actions, providing new results for both single-player and two-player scenarios.

Contribution

It establishes the existence of uniform values and optimal strategies in commutative stochastic games, including deterministic and Lipschitz cases, extending previous results to broader settings.

Findings

Existence of uniform value in commutative stochastic games.

Pure strategies can be 0-optimal in single-player deterministic cases.

Uniform value and equilibrium exist in Lipschitz deterministic two-player games.

Abstract

We are interested in the convergence of the value of n-stage games as n goes to infinity and the existence of the uniform value in stochastic games with a general set of states and finite sets of actions where the transition is commutative. This means that playing an action profile a 1 followed by an action profile a 2 , leads to the same distribution on states as playing first the action profile a 2 and then a 1. For example, absorbing games can be reformulated as commutative stochastic games. When there is only one player and the transition function is deterministic, we show that the existence of a uniform value in pure strategies implies the existence of 0-optimal strategies. In the framework of two-player stochastic games, we study a class of games where the set of states is R m and the transition is deterministic and 1-Lipschitz for the L 1-norm, and prove that these games have a uniform value. A similar proof shows the existence of an equilibrium in the non zero-sum case. These results remain true if one considers a general model of finite repeated games, where the transition is commutative and the players observe the past actions but not the state.