Stationary Markov Nash equilibria for nonzero-sum constrained ARAT Markov games

TL;DR

This paper proves the existence of stationary Markov Nash equilibria in nonzero-sum constrained Markov games with abstract state spaces, under certain regularity and additivity conditions.

Contribution

It establishes the existence of constrained stationary Markov Nash equilibria in nonzero-sum Markov games with ARAT conditions and measurable state spaces, extending previous results.

Findings

Existence of stationary Markov Nash equilibria under ARAT conditions.

Equilibrium strategies satisfy constraints on discounted payoffs.

Results applicable to abstract measurable state spaces.

Abstract

We consider a nonzero-sum Markov game on an abstract measurable state space with compact metric action spaces. The goal of each player is to maximize his respective discounted payoff function under the condition that some constraints on a discounted payoff are satisfied. We are interested in the existence of a Nash or noncooperative equilibrium. Under suitable conditions, which include absolute continuity of the transitions with respect to some reference probability measure, additivity of the payoffs and the transition probabilities (ARAT condition), and continuity in action of the payoff functions and the density function of the transitions of the system, we establish the existence of a constrained stationary Markov Nash equilibrium, that is, the existence of stationary Markov strategies for each of the players yielding an optimal profile within the class of all history-dependent profiles.