A Zero-Sum Stochastic Game with Compact Action Sets and no Asymptotic Value

TL;DR

This paper presents an example of a zero-sum stochastic game with continuous payoffs and transitions where the discounted and n-stage game values fail to converge, challenging assumptions about game stability.

Contribution

It provides the first known example of such a game with compact action sets and no asymptotic value, highlighting limitations in existing convergence theories.

Findings

Discounted game values do not converge as discount factor approaches zero.

n-stage game values do not converge as number of stages increases.

The example involves a four-state game with continuous functions.

Abstract

We give an example of a zero-sum stochastic game with four states, compact action sets for each player, and continuous payoff and transition functions, such that the discounted value does not converge as the discount factor tends to 0, and the value of the n-stage game does not converge as n goes to infinity.