A Potential Reduction Algorithm for Two-person Zero-sum Mean Payoff Stochastic Games

TL;DR

This paper introduces a new finite-time algorithm for two-player zero-sum stochastic games that determines approximate stationary strategies or identifies significant value differences between initial positions, strengthening existing ergodicity results.

Contribution

The paper presents a novel potential transformation-based algorithm that provides constructive guarantees for $ ext{epsilon}$-ergodicity in stochastic games, improving upon prior existential results.

Findings

Algorithm guarantees $ ext{epsilon}$-ergodic strategies in finite time.

Identifies initial positions with value differences of at least $ ext{epsilon}/24$.

Strengthens the connection between $ ext{epsilon}$-ergodicity and stationary strategies.

Abstract

We suggest a new algorithm for two-person zero-sum undiscounted stochastic games focusing on stationary strategies. Given a positive real $\epsilon$, let us call a stochastic game $\epsilon$-ergodic, if its values from any two initial positions differ by at most $\epsilon$. The proposed new algorithm outputs for every $\epsilon>0$ in finite time either a pair of stationary strategies for the two players guaranteeing that the values from any initial positions are within an $\epsilon$-range, or identifies two initial positions $u$ and $v$ and corresponding stationary strategies for the players proving that the game values starting from $u$ and $v$ are at least $\epsilon/24$ apart. In particular, the above result shows that if a stochastic game is $\epsilon$-ergodic, then there are stationary strategies for the players proving $24\epsilon$-ergodicity. This result strengthens and provides a constructive version of an existential result by Vrieze (1980) claiming that if a stochastic game is $0$-ergodic, then there are $\epsilon$-optimal stationary strategies for every $\epsilon > 0$. The suggested algorithm is based on a potential transformation technique that changes the range of local values at all positions without changing the normal form of the game.