Nash Equilibrium Approximation under Communication and Computation Constraints in Large-Scale Non-cooperative Games

TL;DR

This paper addresses the challenge of approximating Nash equilibria in large-scale mean-field games with communication and computation constraints, proposing methods that reduce costs while maintaining accuracy.

Contribution

It introduces three novel approximation methods for the asymptotic equilibrium mean in constrained large-scale games, with theoretical analysis and numerical validation.

Findings

Convergence of equilibrium mean variables to a constant as agents increase

Proposed methods significantly reduce communication and computation costs

Numerical examples demonstrate the accuracy of the approximation methods

Abstract

This paper studies the problem of Nash equilibrium approximation in large-scale heterogeneous mean-field games under communication and computation constraints. A deterministic mean-field game is considered in which the non-linear utility function of each agent depends on its action, the average of other agents' actions (called the mean variable of that agent) and a deterministic parameter. It is shown that the equilibrium mean variables of all agents converge uniformly to a constant, called asymptotic equilibrium mean (AEM), as the number of agents tends to infinity. The AEM, which depends on the limit of empirical distribution of agents' parameters, determines the asymptotic equilibrium behavior of agents. Next, the problem of approximating the AEM at a processing center under communication and computation constraints is studied. Three approximation methods are proposed to substantially reduce the communication and computation costs of approximating AEM at the processing center. The accuracy of the proposed approximation methods is analyzed and illustrated through numerical examples.

Equations

$$z_{i,n}=\frac{1}{n-1}\sum _{j\ne i}{x}_j$$

$$\underset{a\le x_i\le b}{\mathrm{maximize}}$$

$$x_{i,n}^{\star }=\arg \underset{a\le x_i\le b}{\max }$$

$$z_{i,n}^{\star }=\frac{1}{n-1}\sum _{j\ne i}{x}_{j,n}^{\star }$$

$$\mathrm{Br}\left(z_{i,n},{\alpha }_i\right)=\arg \underset{a\le x_i\le b}{\max }$$

$$x_{i,n}^{\star }=\mathrm{Br}\left(z_{i,n}^{\star },{\alpha }_i\right),1\le i\le n.$$

$$\left|\mathrm{Br}\left(z,\alpha \right)-\mathrm{Br}\left(z^{\prime },\alpha \right)\right|\le L_z\left|z-z^{\prime }\right|\forall z,z^{\prime },\alpha \in \mathbb{R}$$

$$\left|\mathrm{Br}\left(z,\alpha \right)-\mathrm{Br}\left(z,{\alpha }^{\prime }\right)\right|\le L\left|\alpha -{\alpha }^{\prime }\right|\forall z,\alpha ,{\alpha }^{\prime }\in \mathbb{R}$$

$$P_n\left(\alpha \right)=\frac{1}{n}\sum _{i=1}^n{1}_{\left\{{\alpha }_i\le \alpha \right\}}.$$

$$\underset{n\to \infty }{\lim }{P}_n\left(\alpha \right)=P\left(\alpha \right)\forall x\in \mathbb{R}.$$

$$z_{\mathrm{AEM}}^{\star }$$

$$={\mathsf{E}}_{\nu }\left[\mathrm{Br}\left(z_{\mathrm{AEM}}^{\star },\nu \right)\right]$$

$$x_{i,\mathrm{ANE}}^{\star }=\mathrm{Br}\left(z_{\mathrm{AEM}}^{\star },{\alpha }_i\right)$$

$$\underset{n\to \infty }{\lim }\underset{1\le i\le n}{\sup }\left|z_{i,n}^{\star }-z_{\mathrm{AEM}}^{\star }\right|=0.$$

$$\underset{n\to \infty }{\lim }{x}_{i,n}^{\star }=x_{i,\mathrm{ANE}}^{\star }.$$

$${\hat{z}}_{\mathrm{AEM}}^{\star }$$

$$={\mathsf{E}}_{\nu }\left[\mathrm{Br}\left({\hat{z}}_{\mathrm{AEM}}^{\star },Q\left(\nu \right)\right)\right]$$

$$\left|z_{\mathrm{AEM}}^{\star }-{\hat{z}}_{\mathrm{AEM}}^{\star }\right|\le \frac{L}{1-L_z}{\mathsf{E}}_{\nu }\left[\left|\nu -Q\left(\nu \right)\right|\right].$$

$${\hat{z}}_{n,\mathrm{AEM}}^{\star }$$

$$={\mathsf{E}}_{{\hat{\nu }}_n}\left[\mathrm{Br}\left({\hat{z}}_{n,\mathrm{AEM}}^{\star },{\hat{\nu }}_n\right)\right]$$

$$\left|{\hat{z}}_{n,\mathrm{AEM}}^{\star }-z_{\mathrm{AEM}}^{\star }\right|\le \frac{L}{1-L_z}{\mathsf{E}}_{\nu ,{\hat{\nu }}_n}\left[\left|\nu -{\hat{\nu }}_n\right|\right].$$

$$\underset{1\le i\le n}{\max }\left|{\hat{z}}_{n,\mathrm{AEM}}^{\star }-z_{i,n}^{\star }\right|=O\left(\frac{1}{n}\right).$$

$${\tilde{\nu }}_n=c_i\text{if ν^n∈Ci}$$

$${\tilde{P}}_n\left(x\right)=\sum _{j=1}^k{1}_{\left\{c_j\le x<c_{j+1}\right\}}\frac{1}{n}\sum _{i=1}^n{1}_{\left\{{\alpha }_i\le \sup C_j\right\}}$$

$${\tilde{z}}_{n,\mathrm{AEM}}^{\star }$$

$$={\mathsf{E}}_{{\tilde{\nu }}_n}\left[\mathrm{Br}\left({\tilde{z}}_{n,\mathrm{AEM}}^{\star },{\tilde{\nu }}_n\right)\right]$$

$$\left|{\tilde{z}}_{n,\mathrm{AEM}}^{\star }-{\hat{z}}_{n,\mathrm{AEM}}^{\star }\right|\le \frac{L}{1-L_z}{\mathsf{E}}_{{\hat{\nu }}_n}\left[\left|{\tilde{\nu }}_n-{\hat{\nu }}_n\right|\right].$$

$$u\left(x_i,z_{i,n},{\alpha }_i\right)=-{\left(x_i-\sqrt{z_{i,n}^2+{\alpha }_i^2}\right)}^2-2x_i.$$

$$Z-Z^{\prime }=\frac{1}{n-1}\left(1_n-I_n\right)\left(X-X^{\prime }\right)$$

$${\Vert G\left(X\right)-G\left(X^{\prime }\right)\Vert }_{\infty }$$

$$\stackrel{(a)}{\le }{L}_z\underset{i}{\max }\left|z_{i,n}-z_{i,n}^{\prime }\right|$$

$$=L_z{\Vert Z-Z^{\prime }\Vert }_{\infty }$$

$$\le L_z{\Vert X-X^{\prime }\Vert }_{\infty }$$

$$\left|F\left(z\right)-F\left(z^{\prime }\right)\right|$$

Full text

I INTRODUCTION

Game theory provides mathematical frameworks for analyzing and predicting the strategic behavior of profit-maximizing agents. Computing the solution of a game is a central research problem in the game theory literature. However, in large population games, it may not be computationally feasible to precisely calculate a game’s solution. Furthermore, even when the computation of solution is possible, it may not provide any insight on the interplay between the equilibrium and population of the game. To understand the equilibrium behavior of large-scale games, which are computationally difficult to solve, it is common to analyze them in the infinite population limit. Despite the asymptotic nature of this method, it provides approximate solutions for large-scale games and sheds light on their equilibrium-population interplay.

Finding an approximate solution in large-scale heterogeneous games, where the agents’ utilities are asymmetric, becomes a cumbersome task as such solutions depend on a large number of parameters. When the approximate solution is computed centrally, e.g., using the cloud computing technology, each agent needs to transmit its parameters to the “cloud” which requires substantial communication and storage resources. The current paper studies the problem of approximating the equilibrium behavior of large-scale games under communication and computation constraints.

I-A Related work

Aumann in [1] showed that in a market with a continuum of traders the core of the market, i.e., the set of allocations in which traders have no motivation for coalition, coincides with the equilibrium allocation, thus, the market becomes perfectly competitive. The authors in [2] studied the value of a voting game with large number of agents. These results were extended to cooperative games in [3]. Haurie and Marcotte in [4] considered a transportation network with $m$ source-destination pairs in which each source-destination pair has $n$ identical agents sharing the same link traversal cost and the same demand function. They showed that as $n$ becomes large the equilibrium flow of links is characterized by a Wardop equilibrium. We note that the existence of pure Nash equilibrium for the non-cooperative games with a continuum of players, i.e, with an uncountable set of agents, has been studied in [5, 6, 7] and [8].

Homogeneous deterministic mean-field games, wherein each agent’s utility depends on the mean of all agents’ states (or control strategies), have been investigated in the literature. The papers [9] and [10] examined the equilibrium behavior of homogenous continuous-time (deterministic) mean-field games as well as their engineering applications. The authors in [11] proposed a mean-field based decentralized equilibrium seeking algorithm for a homogenous discrete-time game with application in the plug-in electric vehicle charging problem. The paper [12] proposed decentralized equilibrium seeking methodologies for linear quadratic mean-field games with convex states. The interested reader is refereed to [13, 14, 15] and references therein for a thorough treatment of the stochastic mean filed games with minor and major agents.

Different from these works, the current paper focuses on the problem of approximating the equilibrium strategies of agents, in heterogeneous, static (deterministic) mean-field games, under communication and computation constraints.

I-B Contributions

This paper considers the problem of approximating the Nash equilibrium of a non-cooperative mean-field game at a processing center. A heterogeneous, deterministic mean-field game is considered in which the (non-linear) utility function of each agent $i$ depends on its action, the average of other agents’ actions (called the mean variable of agent $i$) and a deterministic parameter $\alpha_{i}$. First, the existence and uniqueness of the Nash equilibrium (NE) of the considered game are studied. It is shown that the equilibrium mean variables of all agents converge uniformly to a constant called asymptotic equilibrium mean (AEM) as the number of agents becomes large. The AEM, which characterizes the asymptotic behavior of each agent’s NE strategy, depends on the limit of (deterministic) empirical distribution of $\left\{\alpha_{i}\right\}_{i}$.

Next, three approximation methods are proposed to reduce the computational/communication complexity of finding the AEM at a processing center. The AEM is the solution of an implicit equation which involves integration with respect to the limiting distribution of $\left\{\alpha_{i}\right\}_{i}$ (see equation (1) for more details). The first approximation method reduces the computational complexity of solving the implicit equation by quantizing the underlying distribution function. Under this method, the processing center requires the knowledge of the limiting distribution of $\left\{\alpha_{i}\right\}_{i}$.

The second method approximates the AEM using the empirical distribution of $\left\{\alpha_{i}\right\}_{i}$. This method is suitable for applications in which the limiting distribution of $\left\{\alpha_{i}\right\}_{i}$ is not known. However, it requires that each agent $i$ to transmit its $\alpha_{i}$ to the processing center which might entail substantial communication and storage costs when the agents’ population is large. Under the third approximation method, the processing center receives the quantized versions of $\left\{\alpha_{i}\right\}_{i}$ and approximates the AEM using the quantized values. The accuracy of the proposed approximation methods is investigated. The proposed approximation methods significantly reduce the (computation/communication) cost of evaluating the equilibrium of large scale heterogeneous games using modern computation technologies, e.g., cloud computing.

The rest of this paper is structured as follows. Section II describes our system model and assumptions. Section III presents our results on the asymptotic equilibrium behavior of agents. Section IV studies the AEM approximation problem under communication and computation constraints. Section V presents the numerical results and Section VI concludes the paper.

II System Model

Consider a static non-cooperative game with $n$ agents. Let $u\left(x_{i},z_{i,n},\alpha_{i}\right)$ denote the utility function and $x_{i}$ the action (decision) of agent $i$. The utility function is parameterized by $\alpha_{i}\in\mathbb{R}$ and

[TABLE]

is the mean variable observed by agent $i$, i.e., the average of actions of all agents except the $i$-the one. The action space of each agent is assumed to be limited to the interval $\left[a,b\right]\subset\mathbb{R}$ as a starting point, which we will generalize to jointly convex and compact constraint sets in the future. We refer to the defined $n$-player non-cooperative game among agents as $\mathcal{G}_{n}$.

Note that the utility functions of agents belong to the parametrized family of functions $\left\{u\left(\cdot,\cdot,\alpha\right)\right\}_{\alpha\in\mathbb{R}}$. The interpretation of the $\alpha$ parameter depends on the underlying application. For example, consider the power control problem in downlink code-division-multiple-access networks in a slow fading environment with non-orthogonal spreading codes and match filter receivers. In this application, the parameter $\alpha_{i}$ may represent the channel power gain between the base station and receiver (agent) $i$.

In non-cooperative games, each agent selfishly maximizes its own utility function, given the actions (strategies). Thus, agent $i$ is interested in the solution of the following optimization problem:

[TABLE]

In this paper, the Nash equilibrium (NE) is considered as the solution of the non-cooperative game among agents. Let $x^{\star}_{i,n}$ be the NE strategy of agent $i$ in the game $\mathcal{G}_{n}$ with $n$ players. At the NE point of the game, no agent has motivation to unilaterally deviate his strategy given the NE strategy of the other agents, i.e.,

[TABLE]

where

[TABLE]

Next, the best response function of each agent is introduced which plays an important role in our final results. The best response of agent $i$ to $z_{i,n}$, which follows from the solution of (1), is defined as

[TABLE]

It is well known that, any intersection of the best response functions of agents is a NE, i.e.,

[TABLE]

This model will be used to characterize the asymptotic behavior of the NE strategies of agents in the game $\mathcal{G}_{n}$ as the number of agents becomes large. In practice, the results obtained in the paper enables a centralised processing center compute the approximate NE of a large-scale game with many agents. The processing center approximately computes the NE based on either a priori information available, e.g. coming from a previous instance of a similar game, or explicit information communicated by the agents to the processing center about their individual utilities, in the form of (quantized) $\alpha_{i}$s. The communication and computational model is visualised in Figure 1. Note that, from a technological perspective, this setup fits well into prevalent cloud computing models.

The following additional assumptions are imposed on the utility functions of agents and the parameters $\left\{\alpha_{i}\right\}_{i}$.

The best response function is Lipschitz in $z$ with the Lipschitz constant $L_{z}<1$, i.e., there exists a positive constant $L_{z}<1$ such that

[TABLE] 2. 2.

The best response function is Lipschitz in the parameter $\alpha$ with the Lipschitz constant $L$, i.e., we can find a positive constant $L$ such that

[TABLE] 3. 3.

The empirical distribution of the deterministic parameters $\left\{\alpha_{i}\right\}_{i}$ converges to a continuous distribution function $P\left(\alpha\right)$ as the number of agents increases. Let $P_{n}\left(\alpha\right)$ be the (deterministic) empirical distribution function of $\left\{\alpha_{i}\right\}_{i=1}^{n}$, i.e.,

[TABLE]

Then, we have

[TABLE]

Assumptions 1 and 2 imply that the best response of each agent $i$ cannot change arbitrarily fast as its mean variable $z_{i,n}$ or its parameter $\alpha_{i}$ vary.

Remark 1

We note that our results in this paper do not entirely depend on the Assumption 3. In the case that the limiting distribution $P\left(\alpha\right)$ is not known or does not exist, we use the empirical distribution function of $P_{n}\left(\alpha\right)$ to study the asymptotic behavior of the NE of the game $\mathcal{G}_{n}$ (see Proposition 3 for more details).

III Equilibrium Analysis

In this section, we first study the existence and uniqueness of the NE of the game $\mathcal{G}_{n}$. We next characterize the asymptotic behavior of NE strategies of agents.

Next theorem establishes the existence and uniqueness of the NE of the game $\mathcal{G}_{n}$.

Lemma 1

For $n\geq 2$, the game $\mathcal{G}_{n}$ admits a unique Nash equilibrium.

Proof:

See Appendix A. ∎

Lemma 1 is proved by showing that the best response function of the game forms a contraction mapping. We next characterize the asymptotic behavior of the (equilibrium) mean variables $z^{\star}_{i,n}$s as the number of agents becomes large. To this end, consider the sequence of games $\left\{\mathcal{G}_{n}\right\}_{n=2}^{\infty}$ with increasing number of agents such that the equation (3) holds. For this sequence of games, we define the asymptotic equilibrium mean (AEM), i.e., $z^{\star}_{\rm AEM}$, which captures the asymptotic behavior of equilibrium mean variables.

Definition 1

Consider the sequence of games $\left\{\mathcal{G}_{n}\right\}_{n=2}^{\infty}$ which satisfies the equation (3). Then, the asymptotic equilibrium mean (AEM) $z^{\star}_{\rm AEM}$ is defined as the solution of following equation

[TABLE]

where $\nu$ is a generic random variable distributed according to $P\left(\alpha\right)$.

The asymptotic behavior of the NE of the game $\mathcal{G}_{n}$ can also be characterized using $z^{\star}_{\rm AEM}$. Next, we define the notion of asymptotic Nash equilibrium (ANE) strategy of each agent.

Definition 2

Let $\left\{\mathcal{G}_{n}\right\}_{n=2}^{\infty}$ be a sequence of games such that the Assumption 3 in Section II holds. Let $z^{\star}_{\rm AEM}$ be the solution of (1). Then, the asymptotic Nash equilibrium (ANE) strategy of the agent $i$, denoted by $x^{\star}_{i,\rm ANE}$, is defined as

[TABLE]

The next lemma studies the uniqueness of $z^{\star}_{\rm AEM}$.

Lemma 2

The equation (1) has a unique solution.

Proof:

See Appendix B. ∎

The next theorem studies the asymptotic behavior of the triangular array of (equilibrium) mean variables $\left\{z^{\star}_{i,n},1\leq i\leq n\right\}_{n}$.

Theorem 1

Consider the sequence of games $\left\{\mathcal{G}_{n}\right\}_{n=2}^{\infty}$ such that the Assumption 3 in Section II holds. Let $z^{\star}_{i,n}$ denote the equilibrium mean variable of the agent $i$ at the NE of the game $\mathcal{G}_{n}$. Then, we have

[TABLE]

Proof:

Please see Appendix C. ∎

According to Theorem 1, the mean variables of agents, at the NE of the game, converge uniformly to $z^{\star}_{\rm AEM}$ as the number of agents becomes large.

The next corollary shows that the NE strategy of each agent converges to its ANE strategy as the number of agents becomes large.

Corollary 1

Let $x^{\star}_{i,n}$ denote the NE strategy of agent $i$ in the game $\mathcal{G}_{n}$. Then, we have

[TABLE]

Proof:

The proof follows directly from Theorem 1 and the continuity of ${\rm Br}\left(z,\alpha_{i}\right)$ in $z$. ∎

IV Approximating AEM at the Processing Center

Based on Theorem 1 and Corollary 1, the AEM, i.e., $z^{\star}_{\rm AEM}$, can be used to approximate the NE strategies of agents in the game $\mathcal{G}_{n}$ as the number of agents becomes large. However, to compute the AEM, the processing center requires $(i)$ the knowledge of the distribution function $P\left(\alpha\right)$, $(ii)$ to solve the implicit equation (1). Even if the distribution function $P\left(\alpha\right)$ is known at the processing center, the expectation on the right hand side of (1) may not have a closed form expression, e.g., when the $P\left(\alpha\right)$ and/or ${\rm Br}\left(z,\alpha\right)$ have sophisticated forms.

In this section, we first propose a method for approximating the AEM when $P\left(\alpha\right)$ is known at the processing center. We then consider the case that the processing center does not know $P_{n}\left(\alpha\right)$ and propose two methods for approximating the AEM using the actual parameter values $\left\{\alpha_{i}\right\}_{i}$. Through this section, it is assumed that the processing center knows the general form of the utility functions.

IV-1 Approximating the AEM by Quantizing $P\left(\alpha\right)$

Assume that the processing center has a priori information on $P\left(\alpha\right)$, which may be due to domain knowledge or prior experiences, e.g. when solving a series of games over time with similar parameters. In such cases, even if the solution is not exact, it can be used for example as a warm starting point as part of a multi-stage solution process. However, finding the exact solution of (1) can be cumbersome when the best response function is non-linear in $\alpha$. Here, a quantization-based approximation method is proposed for computing an approximation of AEM. The proposed method replaces the integration of the best response function (with respect to the continuous distribution $P\left(\alpha\right)$) by a summation to ensure computational feasibility.

Let $Q\left(\cdot\right)$ denote a quantization scheme for $P\left(\alpha\right)$. Further, let ${\rm B}_{m}$ and $q_{m}$ denote the $m$th quantization cell and its corresponding cell center under the quantization scheme $Q\left(\cdot\right)$. Next, we define an approximation of the AEM which is computed using the quantization scheme $Q\left(\cdot\right)$.

Definition 3

Let $Q\left(\cdot\right)$ be a quantization scheme for $P\left(\alpha\right)$. Then, $Q\left(\cdot\right)$-based approximation of ${z}^{\star}_{\rm AEM}$, denoted by $\hat{z}^{\star}_{\rm AEM}$, is defined as the solution of the following equation

[TABLE]

where $\nu$ is a random variable with the distribution $P\left(\alpha\right)$.

Let the map $\hat{F}\left(z\right)$ be defined as $\hat{F}\left(z\right)=\mathsf{E}_{\nu}\left[{\rm Br}\left(z,Q\left(\nu\right)\right)\right]$. Similar to the proof of Lemma 2, it can be shown that $\hat{F}\left(z\right)$ is a contraction and admits a unique fixed point. Thus, $\hat{z}^{\star}_{\rm AEM}$ can be computed using fixed point iterations. Next Proposition derives an upper bound on the distance between the $\hat{z}^{\star}_{\rm AEM}$ and ${z}^{\star}_{\rm AEM}$

Proposition 1

Let $Q\left(\cdot\right)$ be a quantization scheme for $P\left(\alpha\right)$ and $\hat{z}^{\star}_{\rm AEM}$ be the $Q\left(\cdot\right)$-based approximation of $z^{\star}_{\rm AEM}$, i.e., the solution of (3). Then, we have

[TABLE]

Proof:

See Appendix E. ∎

Proposition 1 establishes an upper bound on the distance between $\hat{z}^{\star}_{\rm AEM}$ and the $z^{\star}_{\rm AEM}$ which depends on the Lipschitz constants $L,L_{z}$, and the quantization error. Thus, according to this proposition, the approximation error can be made arbitrarily small by increasing the number of quantization levels, which trades-off computational requirements (memory, processing power) with accuracy.

IV-2 Approximating the AEM Using $P_{n}\left(\alpha\right)$

The $Q\left(\cdot\right)$-based approximation of $z^{\star}_{\rm AEM}$ requires that the knowledge of the distribution function $P\left(\alpha\right)$ to be available at the processing center before solving the game. However, in many applications, the distribution $P\left(\alpha\right)$ may not be known a priori. Here, an alternative method is proposed which approximates $z^{\star}_{\rm AEM}$ using the empirical distribution of $\left\{\alpha_{i}\right\}_{i}$, based on the information received by the processing center from individual agents. To this end, it is assumed that each agent $i$ transmits its parameter $\alpha_{i}$ to the the processing center. Note that, as a starting point, it is assumed here that the agents send real-valued $\alpha$ values. Upon receiving $\left\{\alpha_{i}\right\}_{i}$, the processing center will compute $P_{n}\left(\alpha\right)$.

Definition 4

Consider the game $\mathcal{G}_{n}$ in which the empirical distribution of $\left\{\alpha_{i}\right\}_{i}$ is given by $P_{n}\left(\alpha\right)$. Then, the $P_{n}\left(\alpha\right)$-based approximation of the $z^{\star}_{\rm AEM}$, denoted by $\hat{z}^{\star}_{n,\rm AEM}$, is defined as the solution of the following equation

[TABLE]

where $\hat{\nu}_{n}$ is an auxiliary random variable distributed over $\left\{\alpha_{i}\right\}_{i=1}^{n}$ according to $P_{n}\left(\alpha\right)$ and independent of $\nu$.

It is straight forward to verify that the solution of (4) is unique and can be found using a fixed point iteration. Note that $\hat{z}^{\star}_{n,\rm AEM}$ depends on $n$ and changes as the number of agents varies. Next Proposition derives an upper bound on the distance between $z^{\star}_{\rm AEM}$ and $\hat{z}^{\star}_{n,\rm AEM}$.

Proposition 2

Let $\hat{z}^{\star}_{n,\rm AEM}$ denote the $P_{n}\left(\alpha\right)$-based approximation of $z^{\star}_{\rm AEM}$. Then, we have

[TABLE]

where $\nu$ and $\hat{\nu}_{n}$ are two independent random variables distributed according to $P\left(\alpha\right)$ and $P_{n}\left(\alpha\right)$, respectively.

Proof:

Please see Appendix F. ∎

According to Proposition 2, the error of the $P_{n}\left(\alpha\right)$-based method is upper bounded by the $L_{1}$ distance between the random variables $\nu$ and $\hat{\nu}_{n}$. Note that $\hat{\nu}_{n}$ converges in distribution to $\nu$ since $P_{n}\left(\alpha\right)$ converges to $P\left(\alpha\right)$ as $n$ becomes large. Thus, under mild regularity assumptions, e.g., uniform integrability of $\left\{\hat{\nu}_{n}\right\}_{n}$, $\hat{\nu}_{n}$ converges in mean to $\nu$ [16]. This implies that the distance between $z^{\star}_{\rm AEM}$ and $\hat{z}^{\star}_{n,\rm AEM}$ will be arbitrarily small as the number of agents becomes large if the sequence of distribution functions $\left\{P_{n}\left(\alpha\right)\right\}_{n}$ is well-behaved.

The next Proposition studies the closeness of $\hat{z}^{\star}_{n,\rm AEM}$ and the triangular array of (equilibrium) mean variables $\left\{z^{\star}_{i,n},1\leq i\leq n\right\}_{n}$.

Proposition 3

Consider the non-cooperative game $\mathcal{G}_{n}$. Then, the distance between the (equilibrium) mean variables $\left\{z^{\star}_{i,n}\right\}_{i}$ and the solution of the $P_{n}\left(\alpha\right)$-based approximation can be uniformly bounded as

[TABLE]

Proof:

See Appendix G. ∎

Proposition 3 implies that, for $n$ large enough, the mean variables $\left\{z^{\star}_{i,n}\right\}_{i}$ of the game $\mathcal{G}_{n}$ can be well approximated with $\hat{z}^{\star}_{n,\rm AEM}$. Note that Proposition 3 is proved without assuming that the sequence of empirical distribution functions $\left\{P_{n}\left(\alpha\right)\right\}_{n}$ is convergent. Thus, according to this proposition, $\hat{z}^{\star}_{n,\rm AEM}$ can be used to obtain an accurate approximation of mean variables even if the limiting distribution $P\left(\alpha\right)$ in unknown or it does not exist, i.e., when the sequence $\left\{P_{n}\left(\alpha\right)\right\}_{n}$ is not convergent.

IV-3 Approximating AEM Using Quantized Prameters

In order to compute the $P_{n}\left(\alpha\right)$-based approximation, the processing center requires the knowledge of $\left\{\alpha_{i}\right\}_{i}$. Since the parameters $\left\{\alpha_{i}\right\}_{i}$ are real valued, transmitting each $\alpha_{i}$ with high accuracy will impose a large communication cost on each agent as well as a large storage cost on the processing center. Further, the total communication (storage) cost of the $P_{n}\left(\alpha\right)$-based method increases linearly with the number of agents. Hence, the $P_{n}\left(\alpha\right)$-based approximation will become impractical in large population games especially when $P_{n}\left(\alpha\right)$ changes regularly with time, e.g., when agents regularly enter or leave the game.

To address this problem, an approximation method is proposed in which the processing center computes an approximate AEM using quantized values of $\left\{\alpha_{i}\right\}_{i}$. Under the proposed method, each agent $i$ transmits the quantized version of its $\alpha_{i}$ to the processing center. Then, the processing center computes an approximation of $z^{\star}_{\rm AEM}$ using the empirical distribution of the quantized $\left\{\alpha_{i}\right\}_{i}$. We refer to this approximation method as $\alpha_{i}$-quantized approximation. To this end, consider a quantization scheme on $\mathbb{R}$ with $k$ quantization cells denoted by $\left\{C_{i}\right\}_{i=1}^{k}$, i.e., $\left\{C_{i}\right\}_{i=1}^{k}$ form a non-overlapping partitioning of the real line. Also, let ${c}_{i}\in C_{i}$ denote the representative of the cell ${C}_{i}$. Under the $\alpha_{i}$-quantized approximation method, the agent $i$ transmits $c_{i}$ to the processing center if $\alpha_{i}$ belongs to $C_{i}$. Thus, each agent only needs to communicate $\log_{2}k$ bits to the processing center in order to convey its quantized parameter. We assume that each $C_{i}$ is right-closed, thus $\sup C_{i}$ belongs to $C_{i}$.

To study the performance of the $\alpha_{i}$-quantized approximation method, it is helpful to introduce two auxiliary random variables $\hat{\nu}_{n}$ and $\tilde{\nu}_{n}$. Let $\hat{\nu}_{n}$ be a random variable distributed over $\left\{\alpha_{i}\right\}_{i=1}^{n}$ according to a uniform distribution. The random variable $\tilde{\nu}_{n}$, which is distributed over $\left\{{c}_{i}\right\}_{i=1}^{k}$, is constructed using $\hat{\nu}_{n}$ as follows

[TABLE]

Note that the distribution function of $\tilde{\nu}_{n}$, denoted as $\tilde{P}_{n}\left(x\right)$, can be written as

[TABLE]

where ${c}_{k+1}=\infty$.

Next the notion of $\alpha_{i}$-quantized approximation of $z^{\star}_{\rm AEM}$ is defined.

Definition 5

The $\alpha_{i}$-quantized approximation of $z^{\star}_{\rm AEM}$, denoted by $\tilde{z}^{\star}_{n,\rm AEM}$, is defined as the solution of the following equation

[TABLE]

where $\sum_{i}\mathsf{1}_{\left\{\alpha_{i}\in{C}_{j}\right\}}$ is the number of agents which transmit $c_{j}$ to the processing center.

To compute $\tilde{z}^{\star}_{n,\rm AEM}$, the processing center requires to store $\left\{c_{j}\right\}_{j=1}^{k}$ and $\left\{\sum_{i=1}^{n}\mathsf{1}_{\left\{\alpha_{i}\in C_{j}\right\}}\right\}_{j=1}^{k}$ which can be stored with at most $k\log_{2}k+k\log_{2}n$ bits. However, to compute $\hat{z}^{\star}_{n,\rm AEM}$, the processing center needs to store $n$ real variables.

Next Proposition studies the closeness of the solutions of the $\alpha_{i}$-quantized and $P_{n}\left(\alpha\right)$-based methods.

Proposition 4

Let $\tilde{z}^{\star}_{n,\rm AEM}$ denote the solution of the $\alpha_{i}$-quantized approximation method. Then, we have

[TABLE]

where the expectation is with respect to the distribution of $\hat{\nu}_{n}$ as $\tilde{\nu}_{n}$ is a function of $\hat{\nu}_{n}$.

Proof:

See Appendix H. ∎

According to the Proposition 4, the distance between the solutions of the $P_{n}\left(\alpha\right)$-based and $\alpha_{i}$-quantized methods is controlled by $L$, $L_{z}$ and the $L_{1}$ distance between the random variables $\hat{\nu}_{n}$ and $\tilde{\nu}_{n}$. Similar to $P_{n}\left(\alpha\right)$-based AEM approximation, the $\alpha_{i}$-quantized AEM approximation method does not impose any assumption on the convergence of $\left\{P_{n}\left(\alpha\right)\right\}_{n}$. Thus, this approximation method can be used to obtain an approximation of the NE when the distribution function $P\left(\alpha\right)$ is unknown or does not exists. We note that one can combine the result of Proposition 4 with that in Propositions 2, to obtain a bound on the difference between $\tilde{z}^{\star}_{n,\rm AEM}$ and AEM.

V Numerical Results

In this section, we consider a non-cooperative game and numerically analyze the proximity of the equilibrium behavior of agents with the asymptotic equilibrium mean (AEM) and the solution of the $P_{n}\left(\alpha\right)$-based approximation method. To this end, consider a non-cooperative game in which the utility function of agent $i$ is given by

[TABLE]

It is assumed that the empirical distribution of the parameters $\left\{\alpha_{i}\right\}_{i}$ converges to the distribution function of a Gaussian random variable with zero mean and of variance 4. The action space of each agent is limited to the interval $\left[0.5,20\right]$.

Fig. 2 shows the Nash equilibrium (NE) strategy of Agent 1 and its corresponding equilibrium mean variable as a function of the number of agents. According to Fig.2, the asymptotic Nash equilibrium strategy of Agent 1 and the AEM characterize the equilibrium behavior of Agent 1 as $n$ becomes large. More precisely, based on this figure, the NE strategy of Agent 1 converges to its asymptotic Nash equilibrium strategy, i.e., $x^{\star}_{1,\rm ANE}$, as the number of agents becomes large. Similarly, the equilibrium mean variable of Agent 1 converges to the AEM as depicted in Fig. 2.

Fig. 3 illustrates the error of the $P_{n}\left(\alpha\right)$-based approximation method, i.e., $\max_{1\leq i\leq n}\left|z^{\star}_{i,n}-\hat{z}^{\star}_{n,\rm AEM}\right|$, with the number of agents. According to this figure, the error of the $P_{n}\left(\alpha\right)$-based approximation method decays to zero as the number of agents becomes large. Moreover, based on this figure, the $P_{n}\left(\alpha\right)$-based approximation method can be used to obtain an accurate (uniform) approximation of the equilibrium mean variables of agents even for moderate number of agents.

VI CONCLUSIONS

In this paper, we studied the equilibrium behavior of agents in a class of large-scale heterogeneous mean-field games in which the utility function of each agent depends on its action, average of other agents’ actions and a fixed parameter. First, it was shown that the mean variables of all agents converge to a constant, named asymptotic equilibrium mean (AEM), as the number of agents becomes large. Then, the problem of approximating the AEM at a processing center was considered and three approximation methods were proposed which allow the processing center to approximate the AEM under limited computational and communication resources.

Appendix A Proof of Lemma 1

Lemma 1 is proved by showing that the best response function of the game $\mathcal{G}_{n}$ forms a contraction map in the infinity norm. To this end, let $X=\left[x_{1},\cdots,x_{n}\right]^{\top}$ and $X^{\prime}=\left[x^{\prime}_{1},\cdots,x^{\prime}_{n}\right]^{\top}$ be two $n$-dimensional vectors. Also, let $Z=\left[z_{i,n}\right]_{i}$, and $Z^{\prime}=\left[z^{\prime}_{i,n}\right]_{i}$ where $z_{i,n}=\frac{1}{n-1}\sum_{j\neq i}x_{j}$ and $z^{\prime}_{i,n}=\frac{1}{n-1}\sum_{j\neq i}x^{\prime}_{j}$. Let $G\left(X\right):\left[a,b\right]^{n}\mapsto\left[a,b\right]^{n}$ be the best response of agents to $X\in\left[a,b\right]^{n}$, i.e., $G\left(X\right)=\left[{\rm Br}\left(z_{i,n},\alpha_{i}\right)\right]_{i}$. Note that ${Z-Z^{\prime}}$ can be written as

[TABLE]

where $\mathbf{1}_{n}$ is an $n$-by-$n$ matrix with all entries equal to one and $I_{n}$ is an $n$-by-$n$ identity matrix. Thus, we have $\left\|Z-Z^{\prime}\right\|_{\infty}\leq\left\|X-X^{\prime}\right\|_{\infty}$. The infinity norm of $G\left(X\right)-G\left(X^{\prime}\right)$ can be upper bounded as

[TABLE]

where $(a)$ follows from the fact that the best response function is Lipschitz in $z$. This implies that the $G\left(X\right)$ is a contraction as $L_{z}<1$, hence, it admits a unique fixed point [17]. Since Nash equilibria are fixed points of the best response function, the game $\mathcal{G}_{n}$ admits a unique NE.

Appendix B Proof of Lemma 2

To prove this lemma, let $F\left(z\right)=\mathsf{E}_{\nu}\left[{\rm Br}\left(z,\nu\right)\right]$. We show that $F\left(z\right):\left[a,b\right]\mapsto\left[a,b\right]$ is a contraction. Note that $\left|F\left(z\right)-F\left(z^{\prime}\right)\right|$ can be upper bounded as

[TABLE]

Thus, the map $F\left(z\right)$ has a unique fixed point.

Appendix C Proof of Theorem 1

First note that $\left|z^{\star}_{i,n}-z^{\star}_{\rm AEM}\right|$ can be upper bounded as

[TABLE]

where $(a)$ follows from the triangle inequality, $(b)$ follows from the fact that $\left|z^{\star}_{i,n}-z^{\star}_{1,n}\right|=\frac{1}{n-1}\left|x^{\star}_{i,n}-x^{\star}_{1,n}\right|$ and $(c)$ follows from the fact that $a\leq x^{\star}_{i,n}\leq b$ for all $i$. Next, we derive an expression for $z^{\star}_{\rm AEM}$. For $\delta>0$, let the set ${B}^{\delta}_{j}$ be defined as ${B^{\delta}_{j}}=\left\{\left(j-1\right)\delta<\nu\leq j\delta\right\}$ and $\bar{\nu}^{\delta}_{j}$ be a point in ${B}^{\delta}_{j}$. Next Lemma establishes an asymptotic expansion for $z^{\star}_{\rm AEM}$ in terms of $\delta$, ${B}^{\delta}_{j}$ and $\bar{\nu}^{\delta}_{j}$.

Lemma 3

For all $\delta>0$, $z^{\star}_{\rm AEM}$ can be written as

[TABLE]

where $\mathsf{Pr}_{\nu}\left(B^{\delta}_{j}\right)=\mathsf{Pr}\left\{\left(j-1\right)\delta<\nu\leq j\delta\right\}$.

Proof:

See Appendix D. ∎

Next, we derive a similar asymptotic expansion for $z^{\star}_{1,n}$. To this end, let $\theta_{n}$ be a random variable distributed according to the empirical distribution of $\left\{\alpha_{i}\right\}_{i=2}^{n}$.

Lemma 4

For all $\delta>0$, $z^{\star}_{1,n}$ can be written as

[TABLE]

where $\mathsf{Pr}_{\theta_{n}}\left(B^{\delta}_{j}\right)=\mathsf{Pr}\left\{\left(j-1\right)\delta<\theta_{n}\leq j\delta\right\}$.

Proof:

See Appendix D. ∎

Combining Lemmas 3 and 4, we have

[TABLE]

The absolute value of the second term on the right hand side of (the last equality in) (C) can be upper bounded as

[TABLE]

where $(a)$ follows from the fact that ${\rm Br}\left(z,\alpha\right)$ is Lipschitz in $z$. Combining equations (C) and (C), we have

[TABLE]

Next lemma studies the asymptotic behavior of the first term in the right hand side of (C).

Lemma 5

We have

[TABLE]

Proof:

See Appendix D. ∎

Combining equations (C), (C) and Lemma 5, we have

[TABLE]

The desired result follows from the fact that the positive constant $\delta$ can be arbitrarily close to zero.

Appendix D Proof of Auxiliary Lemmas

D-A Proof of Lemma 3

Note that $z^{\star}_{\rm AEM}$ can be written as

[TABLE]

where $(a)$ follows from Lebesgue dominated convergence Theorem. The second term in the right hand side of (D-A) can be upper bounded as

[TABLE]

where $(a)$ follows from the fact that ${\rm Br}\left(z,\alpha\right)$ is Lipschitz in $\alpha$ and $(b)$ follows from the fact that $\left|\nu-\bar{\nu}^{\delta}_{j}\right|\leq\delta$ when $\nu$ belongs to $B^{\delta}_{j}$. Combining equations (D-A) and (D-A), we have

[TABLE]

D-B Proof of Lemma 4

We can write $z^{\star}_{1,n}$ as

[TABLE]

The absolute value of the second term on the right hand side of (D-B) can be upper bounded as

[TABLE]

where $(a)$ follows from the fact that ${\rm Br}\left(z,\alpha\right)$ is Lipschitz in $z$, $(b)$ follows from the fact that $\left|z^{\star}_{i,n}-z^{\star}_{1,n}\right|=\frac{1}{n-1}\left|x^{\star}_{i,n}-x^{\star}_{1,n}\right|$ and $(c)$ follows from the fact that $a\leq x^{\star}_{i,n}\leq b$ for all $i$. Thus, we have

[TABLE]

Let $\theta_{n}$ be a random variable distributed according to the empirical distribution of $\left\{\alpha_{i}\right\}_{i=2}^{n}$, i.e., a uniform distribution over $\left\{\alpha_{i}\right\}_{i=2}^{n}$. Then, we have

[TABLE]

Following similar steps as those in the Appendix D-A, we have

[TABLE]

where $\mathsf{Pr}_{\theta_{n}}\left(B^{\delta}_{j}\right)=\mathsf{Pr}\left\{(j-1)\delta<\theta_{n}\leq j\delta\right\}$.

D-C Proof of Lemma 5

Let $\theta_{n}$ be a random variable distributed according to the empirical distribution of $\left\{\alpha_{i}\right\}_{i=2}^{n}$. We define two axillary random variables $\gamma$ and $\gamma_{n}$ , distributed over $\left\{\bar{\nu}^{\delta}_{j}\right\}_{j=-\infty}^{\infty}$, with the following probability mass functions:

[TABLE]

Thus, we have

[TABLE]

The distribution functions of $\gamma$ and $\gamma_{n}$ can be written as

[TABLE]

Since $P_{n}\left(\alpha\right)$ converges in distribution to $P\left(\alpha\right)$, $\theta_{n}$ also converges in distribution to $P\left(\alpha\right)$. Recall that $\nu$ is distributed according to $P\left(\alpha\right)$. The distribution function $P\left(\alpha\right)$ is continuous, thus, we have

[TABLE]

as $\theta_{n}$ converges in distribution to $P\left(\alpha\right)$ which implies that $\gamma_{n}$ converges in distribution to $\gamma$. Since ${\rm Br}\left(z,\alpha\right)$ is bounded and continuous in $\alpha$, and $\gamma_{n}$ converges in distribution to $\gamma$, we have

[TABLE]

[16] which completes the proof.

Appendix E Proof of Proposition 1

Note that $\left|z^{\star}_{\rm AEM}-\hat{z}^{\star}_{\rm AEM}\right|$ can be upper bounded as

[TABLE]

where $(a)$ follows from the fact that the best response function is Lipschitz in $z$ and $\alpha$. The desired result follows from the last inequality above.

Appendix F Proof of Proposition 2

To prove this result, we rewrite $\left|\hat{z}^{\star}_{n,\rm AEM}-{z}^{\star}_{\rm AEM}\right|$ as

[TABLE]

where $\nu$ and $\hat{\nu}_{n}$ are two random variables distributed according to $P\left(\alpha\right)$ and $P_{n}\left(\alpha\right)$, receptively. The desired result follows from similar steps as those in the proof of Proposition 1.

Appendix G Proof of Proposition 3

The distance between $z^{\star}_{i,n}$ and $\hat{z}^{\star}_{n,\rm AEM}$ can be upper bounded as

[TABLE]

Note that $a\leq{\rm Br}\left(z,\alpha\right)\leq b$. Also, $\left|z^{\star}_{j,n}-z^{\star}_{i,n}\right|$ can be upper bounded as

[TABLE]

as $a\leq x_{i}\leq b$. Thus, $\left|z^{\star}_{i,n}-\hat{z}^{\star}_{n,\rm AEM}\right|$ can be further upper bounded as

[TABLE]

Hence, we have

[TABLE]

which completes the proof.

Appendix H Proof of Proposition 4

Note that the $P_{n}\left(\alpha\right)$-based AEM can be written as

[TABLE]

Thus, we have

[TABLE]