Limit Value in Zero-Sum Stochastic Games with Vanishing Stage Duration and Public Signals

TL;DR

This paper investigates the asymptotic behavior of zero-sum stochastic games with public signals as the discount factor and stage duration tend to zero, revealing differences between perfect and imperfect observation scenarios.

Contribution

It demonstrates that in stochastic games with public signals, the limit value may not exist in discrete time but does exist in continuous time, highlighting a novel distinction from perfect observation cases.

Findings

Limit value does not exist for stage duration 1 as discount factor approaches 0.

Limit value exists when both stage duration and discount factor tend to 0.

Difference between perfect and imperfect observation cases in asymptotic behavior.

Abstract

We consider the behaviour of $\lambda$-discounted zero-sum games as the discount factor $\lambda$ approaches 0 (that is, the players are more and more patient), in the context of games with stage duration. In stochastic games with stage duration h, players act at times 0, h, 2h, and so on. The payoff and leaving probabilities are proportional to h. When h tends to 0, such discrete-time games approximate games played in continuous time. The asymptotic behavior of the values (when both $\lambda$ and h tend to 0) was already studied in the case of stochastic games with perfect observation of the state and in the state-blind case.We consider the same question for the case of stochastic games with imperfect observation of the state. More precisely, we consider a particular case of such games, stochastic games with public signals, in which players are given at each stage a public signal that depends only on the current state. Our main result states that there exists a stochastic game with public signals, with no limit value (as the discount factor $\lambda$ goes to 0) if stage duration is 1, but with a limit value when stage duration h and discount factor $\lambda$ both tend to 0. Informally speaking, it means that the limit value in discrete time does not exist, but the limit value in continuous time (i.e. when h approaches 0) exists. Such a situation is impossible in the case of stochastic games with perfect observation of the state.