Blackwell-Nash Equilibrium for Discrete and Continuous Time Stochastic Games

TL;DR

This paper investigates the existence of Blackwell-Nash equilibria in both discrete and continuous time stochastic games, identifying conditions under which such equilibria exist or do not exist, especially focusing on single controller additive reward cases.

Contribution

It demonstrates that additive reward conditions are necessary for stationary BNE existence in general stochastic games and introduces conditions ensuring BNE in continuous time scenarios.

Findings

Stationary BNE exists under additive reward conditions in discrete time games.

Counterexamples show non-existence of stationary BNE without additive reward conditions.

In continuous time, stationary deterministic BNE exists for SC-AR stochastic games.

Abstract

We consider both discrete and continuous time finite state-action stochastic games. In discrete time stochastic games, it is known that a stationary Blackwell-Nash equilibrium (BNE) exists for a single controller additive reward (SC-AR) stochastic game which is a special case of a general stochastic game. We show that, in general, the additive reward condition is needed for the existence of a BNE. We give an example of a single controller stochastic game which does not satisfy additive reward condition. We show that this example does not have a stationary BNE. For a general discrete time discounted stochastic game we give two different sets of conditions and show that a stationary Nash equilibrium that satisfies any set of conditions is a BNE. One of these sets of conditions weakens a set of conditions available in the literature. For continuous time stochastic games, we give an example that does not have a stationary BNE. In fact, this example is a single controller continuous time stochastic game. Then, we introduce a continuous time SC-AR stochastic game. We show that there always exists a stationary deterministic BNE for continuous time SC-AR stochastic game. For a general continuous time discounted stochastic game we give two different sets of conditions and show that a Nash equilibrium that satisfies any set of conditions is a BNE.