A mathematical programming based characterization of Nash equilibria of some constrained stochastic games

TL;DR

This paper characterizes Nash equilibria in certain constrained stochastic games using mathematical programming, providing a bridge between game theory and optimization, and extends existing results to more general game classes.

Contribution

It introduces a novel characterization of Nash equilibria in constrained stochastic games via mathematical programs, including non-convex quadratic programs, and discusses computational approaches.

Findings

Nash equilibria correspond to global minima of specific mathematical programs.

Reductions to non-convex quadratic programs under certain independence conditions.

Extension of previous results from zero-sum to more general constrained stochastic games.

Abstract

We consider two classes of constrained finite state-action stochastic games. First, we consider a two player nonzero sum single controller constrained stochastic game with both average and discounted cost criterion. We consider the same type of constraints as in [1], i.e., player 1 has subscription based constraints and player 2, who controls the transition probabilities, has realization based constraints which can also depend on the strategies of player 1. Next, we consider a N -player nonzero sum constrained stochastic game with independent state processes where each player has average cost criterion as discussed in [2]. We show that the stationary Nash equilibria of both classes of constrained games, which exists under strong Slater and irreducibility conditions [3], [2], has one to one correspondence with global minima of certain mathematical programs. In the single controller game if the constraints of player 2 do not depend on the strategies of the player 1, then the mathematical program reduces to the non-convex quadratic program. In two player independent state processes stochastic game if the constraints of a player do not depend on the strategies of another player, then the mathematical program reduces to a non-convex quadratic program. Computational algorithms for finding global minima of non-convex quadratic program exist [4], [5] and hence, one can compute Nash equilibria of these constrained stochastic games. Our results generalize some existing results for zero sum games [1], [6], [7].