General limit value in zero-sum stochastic games

TL;DR

This paper extends the understanding of the existence of the limit value in zero-sum stochastic games, showing that previous results are tight and highlighting limitations in generalizing these results to broader payoff evaluations.

Contribution

It generalizes Bewley and Kohlberg's result to more payoff evaluations and provides counterexamples that limit the extension of Mertens and Neyman's uniform value result.

Findings

Generalized the existence of the asymptotic value to broader payoff evaluations.

Provided counterexamples showing the limits of extending existing results.

Analyzed the special case of absorbing games.

Abstract

Bewley and Kohlberg (1976) and Mertens and Neyman (1981) have proved, respectively, the existence of the asymptotic value and the uniform value in zero-sum stochastic games with finite state space and finite action sets. In their work, the total payoff in a stochastic game is defined either as a Cesaro mean or an Abel mean of the stage payoffs. This paper presents two findings: first, we generalize the result of Bewley and Kohlberg to a more general class of payoff evaluations and we prove with a counterexample that this result is tight. We also investigate the particular case of absorbing games. Second, for the uniform approach of Mertens and Neyman, we provide another counterexample to demonstrate that there is no natural way to generalize the result of Mertens and Neyman to a wider class of payoff evaluations.