Deep Nash and Sequential Mean-Field Equilibria in Cooperative and Non-cooperative Games with Imperfect Information Structures

TL;DR

This paper introduces deep Nash and sequential mean-field equilibria for complex stochastic games with imperfect information, leveraging neural network-like dynamics and providing robust strategies for multi-agent decision-making.

Contribution

It develops new equilibrium concepts for large-scale stochastic games with imperfect information, including deep Nash and approximate sequential mean-field equilibria, extending to multiple sub-populations.

Findings

Deep Nash equilibrium dynamics resemble convolutional neural networks.

Sequential mean-field equilibrium approximates deep Nash with performance convergence.

Strategies are robust to trembling-hand imperfections at multiple levels.

Abstract

A class of nonzero-sum stochastic dynamic games with imperfect information structure is investigated. The game involves an arbitrary number of players, modeled as homogeneous Markov decision processes, aiming to find a sequential Nash equilibrium. The players are coupled in both dynamics and cost functions through the empirical distribution of states and actions of players. Two non-classical information structures are considered: deep state sharing and no-sharing, where deep state refers to the empirical distribution of the states of players. In the former, each player observes its local state as well as the deep state while in the latter each player observes only its local state. For both finite- and infinite-horizon cost functions, a sequential equilibrium, called deep Nash equilibrium, is identified, where the dynamics of deep state resembles a convolutional neural network. In addition, an approximate sequential equilibrium, called sequential mean-field equilibrium, under no-sharing information structure is proposed, whose performance converges to that of the deep Nash equilibrium despite the fact that the strategy is not necessarily continuous with respect to the deep state. The proposed strategies are robust to trembling-hand imperfection in both microscopic and macroscopic levels. Finally, the extension to multiple sub-populations and arbitrarily-coupled (asymmetric) cost functions are demonstrated.