Hidden Stochastic Games and Limit Equilibrium Payoffs

TL;DR

This paper investigates the behavior of equilibrium payoffs in stochastic games with perfect and imperfect state observations as players become more patient, revealing cases where limits of payoffs do not exist.

Contribution

It introduces the concept of hidden stochastic games with imperfect signals and demonstrates the non-existence of equilibrium payoff limits in such models, extending prior results.

Findings

Stationary equilibrium payoffs always converge in standard stochastic games.

In hidden stochastic games, the equilibrium payoff set may not have a limit.

The non-convergence result is robust to various perturbations and communication devices.

Abstract

We consider 2-player stochastic games with perfectly observed actions, and study the limit, as the discount factor goes to one, of the equilibrium payoffs set. In the usual setup where current states are observed by the players, we show that the set of stationary equilibrium payoffs always converges, and provide a simple example where the set of equilibrium payoffs has no limit. We then introduce the more general model of hidden stochastic game, where the players publicly receive imperfect signals over current states. In this setup we present an example where not only the limit set of equilibrium payoffs does not exist, but there is no converging selection of equilibrium payoffs. This second example is robust in many aspects, in particular to perturbations of the payoffs and to the introduction of correlation or communication devices.