Nonstationary nonzero-sum Markov games under a probability criterion

TL;DR

This paper studies nonstationary nonzero-sum Markov games focusing on maximizing the probability of achieving certain reward thresholds, providing existence proofs for Nash equilibria and an algorithm for their computation.

Contribution

It introduces a probability-based criterion for nonstationary Markov games, proving Nash equilibrium existence and offering an efficient epsilon-Nash equilibrium algorithm.

Findings

Proved existence of Nash equilibria under probability criterion.

Developed an efficient algorithm for epsilon-Nash equilibria.

Demonstrated results with a nonstationary energy management example.

Abstract

This paper deals with N-person nonzero-sum discrete-time Markov games under a probability criterion, in which the transition probabilities and reward functions are allowed to vary with time. Differing from the existing works on the expected reward criteria, our concern here is to maximize the probabilities that the accumulated rewards until the first passage time to any target set exceed a given goal, which represent the reliability of the players income. Under a mild condition, by developing a comparison theorem for the probability criterion, we prove the existence of a Nash equilibrium over history-dependent policies. Moreover, we provide an efficient algorithm for computing epsilon-Nash equilibria. Finally, we illustrate our main results by a nonstationary energy management model and take a numerical experiment.