Strategic Prediction with Latent Aggregative Games

TL;DR

This paper presents a novel class of structured prediction models based on latent aggregative games that capture strategic interactions and can recover meaningful behaviors from real voting data.

Contribution

It introduces a new game-based modeling framework with convergent dynamics and identifiability conditions, enabling transferability across different sets of players.

Findings

Successfully recover strategic interactions from voting data

Propose two new aggregator paradigms with convergence guarantees

Model demonstrates transferability across different scenarios

Abstract

We introduce a new class of context dependent, incomplete information games to serve as structured prediction models for settings with significant strategic interactions. Our games map the input context to outcomes by first condensing the input into private player types that specify the utilities, weighted interactions, as well as the initial strategies for the players. The game is played over multiple rounds where players respond to weighted aggregates of their neighbors' strategies. The predicted output from the model is a mixed strategy profile (a near-Nash equilibrium) and each observation is thought of as a sample from this strategy profile. We introduce two new aggregator paradigms with provably convergent game dynamics, and characterize the conditions under which our games are identifiable from data. Our games can be parameterized in a transferable manner so that the sets of players can change from one game to another. We demonstrate empirically that our games as models can recover meaningful strategic interactions from real voting data.

Equations

$$U_i({\sigma }_i,{\sigma }_{-i},z_i)={\sigma }_i^{\top }\left({\mathcal{A}}_i({\sigma }_{-i})-z_i\right)+\tau \mathbb{H}({\sigma }_i)\text{leavevmode},$$

$${\beta }_i^{\tau }({\mathcal{A}}_i({\sigma }_{-i}),z_i)\in \arg \underset{{\sigma }_i\in △(A_i)}{\max }{U}_i({\sigma }_i,{\sigma }_{-i},z_i)\text{leavevmode}.$$

$$U_i({\sigma }_i^{\ast },{\sigma }_{-i}^{\ast },z_i)\text{leavevmode}\text{leavevmode}\ge \text{leavevmode}\text{leavevmode}{U}_i({\sigma }_i,{\sigma }_{-i}^{\ast },z_i)\text{leavevmode}\forall i\in [n],{\sigma }_i\in △(A)\text{leavevmode}.$$

$${\hat{\sigma }}_i^{t+1}(x^{(m)})=\zeta (\nu ({\hat{\sigma }}_i^t)+\alpha ({\hat{\mathcal{A}}}_i({\hat{\sigma }}_{-i}^t)-{\hat{z}}_i^{(m)})),\text{leavevmode}\text{leavevmode}\text{leavevmode}t=0,1,\dots ,k-1,$$

$${\mathcal{A}}_{x,v,\mathcal{I}}({\sigma }_{x,-v})\triangleq \sum _{v^{\prime }\in \mathcal{I}:v^{\prime }\ne v}{w}_{x,v,v^{\prime }}{\sigma }_{x,v^{\prime }}\text{leavevmode}.$$

$$U_{v,\mathcal{I}}({\sigma }_{x,v},{\sigma }_{x,-v},z_{x,v})={\sigma }_v^{\top }\left({\mathcal{A}}_{x,v,\mathcal{I}}({\sigma }_{x,-v}))-\Gamma z_{x,v}\right).$$

$$q_i^k$$

$$h_j(q_j(t))$$

$$(\text{LAG-FP})\text{AA yields }{\overline{u}}^k:\text{leavevmode}{g}_i({\overline{u}}^k,z_i)$$

$$\text{PA yields }{u}^k:\text{leavevmode}{g}_i(u^k,z_i)$$

$$(\text{LAG-GP})\text{AA yields }{\overline{u}}^k:\text{leavevmode}\text{leavevmode}{g}_i({\overline{u}}^k,z_i)$$

$$\text{PA yields }{u}^k:\text{leavevmode}\text{leavevmode}{g}_i(u^k,z_i)$$

$$(\text{LAG-FP/AA})\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}{\dot{q}}_i$$

$$(\text{LAG-FP/ PA})\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}{\dot{q}}_i$$

$$(\text{LAG-GP/AA})\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}{\dot{q}}_i$$

$$(\text{LAG-GP/PA})\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}{\dot{q}}_i$$

$$\left[\begin{array}{cc}(1+\gamma \lambda )W& -\gamma \lambda W\\ \lambda W& -\lambda I\end{array}\right].$$

$$\arg \underset{w_i\in {\mathbb{R}}^{n-1}}{\min }{\ell }_i(w_i;D)\text{leavevmode}+\text{leavevmode}\lambda ||w_i|{|}_1\text{leavevmode},$$

$${\Lambda }_{\min }\left(H_{i,SS}^{\ast M}\right)$$

$${\Lambda }_{\max }\left(\frac{1}{M}\sum _{m=1}^M{\Phi }_{-i}^{(m)}{\Phi }_{-i}^{(m{)}^{\top }}\right)$$

$$\left[\begin{array}{cc}-I+(1+\gamma \lambda )\mathcal{D}& -\gamma \lambda \mathcal{D}\\ \lambda I& -\lambda I\end{array}\right].$$

$${\beta }_i^{\tau }({\mathcal{A}}_i({\sigma }_{-i}),z_i)=\zeta \left(\frac{{\sum }_{j\ne i}{w}_{ij}{\sigma }_j-z_i}{\tau }\right)=\zeta \left(\frac{{\mathcal{A}}_i({\sigma }_{-i})-z_i}{\tau }\right),$$

$$(\zeta(x))_{\ell}=\exp(x_{\ell})\bigg{/}\sum_{k}\exp(x_{k}).$$

$${\dot{q}}_i$$

$${\dot{r}}_i$$

$$N^{\top }N=I_{m-1},\text{ and }{1}_m^{\top }N=0_{m-1},$$

$$q_i(t)=q_i^{\ast }+N\delta x_{q_i}(t),$$

$$(q_1(t)-q_1^{\ast },\dots ,q_n(t)-q_n^{\ast },\text{leavevmode}\text{leavevmode}\text{leavevmode}{r}_1(t)-q_1^{\ast },\dots ,r_n(t)-q_n^{\ast }{)}^{\top }\text{leavevmode}=\text{leavevmode}\mathcal{N}\delta x(t)\text{leavevmode},$$

$$\delta x(t)=(\delta x_{q_1}(t),\dots ,\delta x_{q_n}(t),\delta x_{r_1}(t),\dots ,\delta x_{r_n}(t){)}^{\top }\in {\mathbb{R}}^{2n(m-1)}$$

$${\mathcal{N}}^{\top }(q_1(t)-q_1^{\ast },\dots ,q_n(t)-q_n^{\ast },r_1(t)-q_1^{\ast },\dots ,r_n(t)-q_n^{\ast }{)}^{\top }\text{leavevmode}=\text{leavevmode}{\mathcal{N}}^{\top }\mathcal{N}\delta x(t)\text{leavevmode}=\text{leavevmode}\delta x(t).$$

$$\dot{\delta }{x}_{q_i}=N^{\top }({\dot{q}}_i-{\dot{q}}_i^{\ast })=N^{\top }{\dot{q}}_i(t)=N^{\top }{F}_i(q_i,q_{-i},r_{-i}).$$

$$F_i(q_i^{\ast },q_{-i}^{\ast },r_{-i}^{\ast })=0_m.$$

$$\approx$$

$$\dot{\delta }{x}_{q_i}=-\delta x_{q_i}\text{leavevmode}+\text{leavevmode}(1+\gamma \lambda )\sum _{k\ne i}{D}_{ik}\delta x_{q_k}\text{leavevmode}-\text{leavevmode}\gamma \lambda \sum _{k\ne i}{D}_{ik}\delta x_{r_k}.$$

Full text

1 Introduction

Structured prediction methods [1, 2, 3, 4] typically operate on parametric scoring functions whose maximizing assignment is used as the predicted configuration. Since the parameters can be learned directly to maximize prediction accuracy, often via surrogate losses, the methods have been successful across areas [5, 6, 7, 8, 9, 10]. However, not all structured observations can be naturally modeled as extrema of scoring functions. For instance, votes on a bill in the US Congress pertain to actions or strategies adopted by individual senators following several rounds of negotiations with a subset of other senators. These votes do not generally correspond to any common scoring function, and should be modeled as an equilibrium rather than a maximizing assignment [11, 12].

Previous work has considered outcomes as pure strategy Nash equilibria (PSNE) of some fixed underlying graphical game [12, 13, 14, 15], where the payoff of a player depends on its own strategy and the (aggregate) strategy of its neighbors in a graph. The advantage of pure strategies is that the observations can be directly related to actions taken in the game. The parameters of the game such as interactions are thus readily adjustable on the basis of observed action profiles. A drawback of this family of models is that PSNE do not exist in a wide class of games [16], and they require players to have sufficient (complete) information about the actions of other players. Moreover, the setup entirely side-steps the issue of game dynamics, i.e., how the equilibrium is arrived at in the game, reducing the ability to use equilibrium as a predicted outcome in a context dependent manner. The key estimation procedures in most PSNE based approaches [12, 13] are also combinatorially hard. From the point of view of applicability, the models are also tailored to a fixed set of players, and thus do not enable inference about the behavior of new players.

We address these issues by extending the scope of structured prediction to games. At a high level, our model takes the available context such as a bill or a resolution to be voted on – the input – and maps it to a predicted outcome which is an action profile. We model the impact of context by parametrically mapping it to private information for each player (player types) which is subsequently incorporated into player utilities and their initial strategy profiles. We adopt mixed strategy Nash equilibria (MSNE) that exist in any game, unlike PSNE. Since player types are hidden, we call our parametrized games Latent Aggregative Games (LAGs). Our games can be viewed as a conditional version of directed graphical games [17, 18], restricted to a rich subclass called aggregative games that subsumes Cournot oligopoly, mean field, public goods, and population games [12]. Aggregative games shield each player from specific information about any neighbor since players respond to aggregate (weighted sum) of their neighbors’ strategies.

A key novelty of our approach is to explicitly incorporate game dynamics that specifies how the predicted equilibrium is derived from the context. In our approach, we follow a $k$-step best response dynamics, seeded with predicted initial strategies, to arrive at a (near) mixed strategy equilibrium. An observed outcome, in response to the context, is then viewed as a sample from this predicted mixed strategy equilibrium. As we operate on continuous strategies and our updates are differentiable functions, we can use standard back-propagation to evaluate gradients through the $k$-step strategy updates, and thus learn parameters efficiently, unlike e.g. [12, 13]. We also generalize strategic prediction to transferable games where the sets of players may change from one game to another. To this end, we incorporate players into the game in terms of their feature representations, and learn a mechanism for mapping these features and the context into payoffs, strategic interactions as well as initial strategies. These games permit us to predict the behavior of new players in new contexts.

Since game dynamics plays a critical role in strategic prediction, we introduce and provide a deeper analysis of more general dynamics and types of aggregation strategies. Our work pins down exact conditions under which strategies converge to different types of equilibrium. Convergence to Nash equilibria has been known largely for a restricted class of games, e.g., two player zero-sum and potential games [19], and multiplayer games are known to be considerably more difficult [20, 21, 22, 23, 24, 25, 26, 27]. Our analysis makes use of tools from control theory, dynamical systems, and stochastic approximation [28, 29, 30, 31, 32, 33], and thus contributes to this line of work as well.

Finally, we provide identifiability guarantees for LAGs. For the analysis, we adopt a simpler one-shot setting, where the observed outcome is sampled from player strategies after one round of communication instead of following $k$-steps to a near mixed strategy equilibrium. We characterize conditions under which one-shot LAGs become identifiable, i.e., we can recover the neighbors of any player with the correct sign of their interaction (positive or negative). Such recovery is infeasible under PSNE since multiple game structures may pertain to the same set of PSNE [14, 34].

2 Basic strategic prediction model

We first introduce our basic strategic prediction model. To this end, we need to define several components of the model. These include (a) the graphical layout of the game, and how players influence each other; (b) the player types (private information) and how these are derived from the context; (c) initial strategies for the players before witnessing the play of others; (d) individual utilities for the players; (e) and the game dynamics, i.e., how players respond to others. Later, in the transferable setting, we will no longer individuate players through their identities but instead adopt feature vectors for players.

Let $G=(E,V)$ be a connected digraph such that vertex $i$ identifies player $i\in[n]\triangleq\{1,2,\ldots,n\}$, where $n=|V|$. Let $A$ be the (common) finite discrete set of actions for all the players, and let $T_{i}\subseteq\mathbb{R}^{|A|}$ be the latent type set for player $i$, defined shortly. Each player $i$ plays a randomized (mixed) strategy which is a distribution over actions: $\sigma_{i}\in\triangle(A)$ such that $\sum_{a_{i}\in A}\sigma_{i}(a_{i})=1$ and $\sigma_{i}(a_{i})\geq 0$ for $a_{i}\in A$. We will denote a joint strategy profile of all the players by $(\sigma_{i},\sigma_{-i})$ to emphasize the distinction between player $i$ and all others. We model the influence of players on others through weighted aggregation of neighbor strategies. The weights $w_{ij}\in\mathbb{R}$ denote the strength of influence of player $j$ on player $i$. We will call players $j\in[n]\setminus\{i\}$ that have $w_{ij}\neq 0$ the neighbors of players $i$. We define a weight matrix $W\in\mathbb{R}^{n\times n}$ such that $W(i,i)=0$ and $W(i,j)=w_{ij}$. Player $i$ communicates with other players only through aggregator $\mathcal{A}_{i}$ that maps the strategies of other players, i.e., $\sigma_{-i}$ to the weighted sum $\sum_{j\neq i}w_{ij}\sigma_{j}$, the effective influence of others. The context influences the game through private types of players. This mapping could be defined in multiple ways. For simplicity, we initially parameterize the private type of each player $i\in[n]$ by a linear transformation or matrix $\theta_{i}:\mathcal{X}\to T_{i}$ that maps context $x\in\mathcal{X}\subseteq\mathbb{R}^{d}$ to $z_{i}(x)=\theta_{i}x$. We will keep the dependence on context implicit, for simplicity, and abbreviate $z_{i}(x)$ as $z_{i}$ from here on when the context is clear. We model the utility or payoff of player $i\in[n]$, of type $z_{i}\in T_{i}$, under strategy profile $(\sigma_{i},\sigma_{-i})$ as:

[TABLE]

where $\mathbb{H}(\sigma_{i})$ is the entropy associated with $\sigma_{i}$ and $\tau\geq 0$. The entropy encourages completely mixed strategy choices, in the interior of simplex $\Delta(A)$. Our payoff functions generalize linear influence games that describe several decision scenarios such as whether to vaccinate against a disease, install antivirus software, or get home insurance [15, 14, 34]. We allow multi-way actions and private types, thus capturing a wider range of strategic behaviors. The payoffs may be interpreted as the expected reward received by players in a repeated game. We can naturally define the best response of player $i$ when it observes the aggregate input $\mathcal{A}_{i}(\sigma_{-i})$ to be

[TABLE]

We say that $(\sigma_{i}^{*},\sigma_{-i}^{*},z_{i},z_{-i})$ forms an MSNE or simply a Nash equilibrium (NE) of LAG iff

[TABLE]

Every finite game has at least one MSNE [35]. We say that MSNE is strict (SNE) when (3) is strict for all $i\in[n],\sigma_{i}\in\Delta(A)\setminus\{\sigma_{i}^{*}\}$, completely mixed (CMNE) when $\sigma_{i}^{*}(a_{i})>0$ for all $i\in[n],a_{i}\in A$, and pure (PSNE) when for all $i\in[n]$ there exists an $a_{i}\in A$ such that $\sigma_{i}^{*}(a_{i})=1$. It remains to specify how an equilibrium is reached, i.e., the game dynamics. To begin with, players observe context $x$, and evaluate types $z_{i}$. Our setting dispenses with the restrictive assumption made by Bayesian games [36, 37, 38] that the conditional distribution $P(z_{-i}|z_{i})$ is known to player $i$. In our case, the types give rise to initial strategies $\sigma_{i}^{0}=\psi(z_{i})$, where $\psi:T_{i}\to\Delta(A)$ (e.g. softmax). The best response dynamics from this point on depends on the details of the aggregator and whether the dynamics is defined over strategies or actions directly. We study several alternative game dynamics with different aggregators in Section 4. In our empirical analysis, we adopt a simpler $k$-step corrective dynamics as described below. Once a (near) equilibrium is reached, a sample action profile $y\in\mathcal{Y}\subseteq A^{n}$ is drawn from player strategies.

Parameter estimation. We learn our games from data as structured prediction methods. Specifically, given a dataset $D=\{(x^{(m)},y^{(m)})\in\mathcal{X}\times\mathcal{Y},m\in[M]\}$ linking contexts to sampled action profiles, our objective is to estimate the type parameters $\theta_{i}$ and the influences of neighbors $w_{i}\triangleq(w_{ij})_{j\neq i},i\in[n]$. Each pair $(x^{(m)},y^{(m)})$ is treated as follows. A linear transformation $\hat{\theta}=(\hat{\theta}_{1},\ldots,\hat{\theta}_{n})$ maps the context $x^{(m)}$ to the types $\hat{z}(x^{(m)})\triangleq(\hat{z}_{1}^{(m)},\ldots,\hat{z}_{n}^{(m)})$ that result in initial strategies $\hat{\sigma}_{i}^{0}(x^{(m)})=\zeta(\hat{z}_{i}^{(m)})$ of the players $i\in[n]$, where $\zeta$ is the softmax nonlinearity. The aggregators $\hat{\mathcal{A}}_{i}$ evaluate weighted sums, and are parametrized by weights $\hat{w}_{i}=(\hat{w}_{ij})_{j\neq i}$. A sequence of $k$ update steps

[TABLE]

is then followed: $k$ and $\alpha$ are hyperparameters, and $\nu$ defines the type of update. Several choices of $\nu$ are possible; e.g., $\nu(\hat{\sigma}_{i}^{t})=0$ pertains to best response $\beta_{i}^{1/\alpha}(\hat{\mathcal{A}}_{i}(\hat{\sigma}_{-i}^{t}),\hat{z}_{i}^{(m)})$ defined in (2), and the identity mapping $\nu(\hat{\sigma}_{i}^{t})=\hat{\sigma}_{i}^{t}$ defines a gradient step. Our estimation criterion for the game is to minimize the expected cross-entropy loss $\mathbb{E}[\ell(\hat{\sigma}^{k}(x^{(m)}),y^{(m)}]$ between the predicted mixed strategies and the observed profiles, where the expectation is with respect to the empirical distribution over pairs $(x^{(m)},y^{(m)})$, $\ell$ is the cross entropy loss, and $\hat{\sigma}^{k}(x^{(m)})\triangleq(\hat{\sigma}_{i}^{k}(x^{(m)}))_{i\in[n]}$. We use standard backpropagation to evaluate gradients through the $k$-step strategy updates efficiently.

3 Transferable strategic prediction

Here we generalize LAGs to permit different players from one game to another. Unlike in section 2, we can no longer assume a fixed interaction structure across games. Instead, the neighbor influences are determined by context and player feature vectors. The model enables us to predict the behavior of new players in new contexts. Specifically, we construct a feature vector $b_{i}\in\mathcal{B}$ for each player $i\in[n]$. Such information is often publicly available, e.g., education and gender of judges; human development indicators of countries, etc. Each game is played with a different subset of players $\mathcal{I}\subseteq[n]$, and is unrolled as follows. A context $x\in\mathcal{X}$ is mapped to latent (more general) player types using a parametric function $f_{z}:\mathcal{X}\times\mathcal{B}\to\mathcal{Z}$, taking each pair $(x,b_{v}),v\in\mathcal{I}$ as input, and mapping it to $z_{x,v}\in\mathcal{Z}$. The latent types define initial strategies as before $\sigma_{x,v}^{0}=\phi(\Gamma z_{x,v})$, where $\Gamma$ is a transformation matrix that yields a vector in $\mathbb{R}^{|A|}$, and $\phi$ (e.g., softmax) maps the result to a distribution in the simplex $\Delta(A)$. Unlike before, the (asymmetric) influences between players are now calculated parametrically from the types: $w_{x,v,v^{\prime}}=f_{w}(z_{x,v},z_{x,v^{\prime}})$ using a parametric mapping $f_{w}:\mathcal{Z}\times\mathcal{Z}\to\mathbb{R}$. Each player $v$ still responds to other players $v^{\prime}\in\mathcal{I}\setminus\{v\}$ through its aggregator

[TABLE]

We can extend the definition of the payoffs slightly to incorporate the more general latent types:

[TABLE]

where $\Gamma$ is an additional parameter matrix to be learned. The game dynamics dictates the course of play in the same fashion as the basic strategic setting. We learn the model, now parameterized by $f_{z}$, $f_{w}$, and $\Gamma$, by minimizing the loss between predicted $k$-step strategies and observed action profiles.

4 General game dynamics and convergence

We now provide an in-depth look at the game dynamics along with associated convergence guarantees. The aggregator in the game acts as a privacy preserving component, hiding specific neighbor actions or strategies, only offering aggregate statistics. We design dynamics under two different kinds of feedback from the aggregator. In an active aggregator (AA) setting, the players get a prediction about the anticipated aggregate of their neighbors. In contrast, a passive aggregator (PA) only provides the aggregate of empirical frequencies used by the neighbors, and changes in the aggregate are estimated by the player. Intuitively, AA reveals more information about the neighbors’ strategy evolution. We devise two new protocols as derivative action adaptations of smooth fictitious play (FP) and gradient play (GP) [39, 40, 41, 42, 43] for aggregative games. The protocols differ by how players respond to the (predicted) aggregate: one can play the best response or adapt the strategy via a gradient update. Formally, player $i$ samples an action $a_{i}^{k}\sim\sigma_{i}^{k}$ at time $k>0$ based on

[TABLE]

where $q_{i}^{k}$ is the empirical frequency of actions played by $i$ till time $k$, and $g_{i}:\mathbb{R}^{|A|}\times\mathbb{R}^{|A|}\to\Delta(A)$ and $h_{i}:\mathbb{R}^{|A|}\to\mathbb{R}^{|A|}$ are appropriately defined Lipschitz mappings possibly involving small input noise. We let $q_{-i}^{k-1}\triangleq\{q_{j}^{k-1}|j\neq i\}$, and $h_{-i}(q_{-i}^{k-1})\triangleq\{h_{j}(q_{j}^{k-1})|j\neq i\}$. We also define the base case $q_{i}^{0}=\sigma_{i}^{0}=\phi(z_{i})$. Note that player $i$ communicates only with $\mathcal{A}_{i}$. We define a passive aggregator (PA) by letting $h_{i}$ be the identity mapping, i.e. $h_{i}(q_{i})=q_{i}$. Alternatively, when $h_{i}(q_{i})=q_{i}+\gamma\nabla\tilde{q}_{i}$ for some $\gamma>0$ and a difference approximation $\nabla\tilde{q}_{i}$ of a temporal derivative $\nabla q_{i}$, we obtain an active aggregator (AA). Intuitively, AA views each $q_{j}$ as discretization of a continuous signal $q_{j}(t)$ so that when $\nabla\tilde{q}_{j}(t)\approx\nabla q_{j}(t)$, for neighbors $j$ of $i$, we have

[TABLE]

and therefore $\mathcal{A}_{i}$ offers a predicted aggregate to player $i$. We consider two forms of best response dynamics encoded in $g_{i}$, LAG-FP and LAG-GP, based on derivative FP and derivative GP, respectively. In LAG-FP we set $\tau>0$ in the utility functions. This lets us have a unique best response:

[TABLE]

where the AA case differs from PA in terms of where the difference approximation happens. In AA, it happens prior to aggregation thus $g_{i}$ is defined directly in terms of the output of AA or $\bar{u}^{k}$ which absorbs any temporal approximation error. In PA, the player constructs a temporal prediction of the aggregate, and the approximation is $(\nabla u^{k}-\nabla\hat{u}^{k})$. In LAG-GP, we set $\tau=0$, and player $i$ takes a gradient step to maximize the anticipated payoff followed by a Euclidean projection to get a unique mapping $g_{i}$ (since any such projection on a closed convex set is unique):

[TABLE]

where $\Pi_{\Delta}(q)\triangleq\arg\!\min_{\tilde{q}\in\triangle(A)}||\tilde{q}-q||_{2}\leavevmode\nobreak\ .$ Thus under both protocols, players take actions stochastically according to $\sigma_{i}^{k}$ and the best response mapping $g_{i}$ is unique for each $k$. Assuming that the error sequence in updating $\{\sigma_{i}^{k}\}$ is a martingale, our updates satisfy the conditions in (section 2.1, [28]) and we can analyze the stochastic evolution of each LAG as a noisy discretization of a limiting ordinary differential equation (ODE). We investigate the conditions under which the fixed points of the this ODE are locally asymptotically stable, and as a consequence, our discrete updates would converge to a Nash equilibrium with positive probability [43]. An equilibrium point $s$ is locally asymptotically stable if every ODE trajectory that starts at a point in a small neighborhood of $s$ remains forever in that neighborhood and eventually converges to $s$.

Our updates in (5) lead to the implicit ODEs (6)-(9) for LAG-FP and LAG-GP under AA and PA settings, where $\lambda>0$, $\dot{r}_{i}$ is an estimate for $\dot{q}_{i}$, and $\dot{r}_{-i}\triangleq\{\dot{r}_{j}|j\neq i,w_{ij}\neq 0\}$. We call a matrix stable if all its eigenvalues have strictly negative real parts. Let $I$ denote the identity matrix. We now specify conditions under which dynamics converge to CMNE or SNE in the PA setting. We defer the convergence to NE in the PA setting, and all convergence results in the AA setting to the supplementary material.

Theorem 1**.**

(LAG-GP/PA convergence to CMNE)* Let the weight matrix $W$ be stochastic. Let $(q_{1}^{*},\ldots,q_{n}^{*},z_{1},\ldots,z_{n})$ be a completely mixed NE under the dynamics in (9). The linearization of (9) with $\gamma>0$ is locally asymptotically stable for $\lambda>0$ if and only if the following matrix is stable*

[TABLE]

Theorem 2**.**

(LAG-GP/PA convergence to SNE)* Let the weight matrix $W$ be doubly stochastic. Let $(q_{1}^{*},\ldots,q_{n}^{*},z_{1},\ldots,z_{n})$ be a strict NE under the dynamics in (9). The equilibrium point $(q_{i}=q_{i}^{*},r_{i}=A_{i}(q_{-i}^{*}))_{i\in[n]}$ is locally asymptotically stable for sufficiently small $\gamma\lambda$, where $\gamma,\lambda>0$.*

Our results have important implications. [12, 13] enforced margin constraints on payoffs in their PSNE setups. They did not establish SNE, which guarantees a strictly worse payoff to any player that unilaterally deviates from equilibrium. Theorem 2 specifies the conditions for convergence to SNE, and dictates when margin constraints should be imposed. The classic fictitious play (FP) fails to converge in simple games, e.g. [42], that have a unique CMNE. Theorem 1 specifies conditions that circumvent such negative results. We prove our results via carefully crafted Lyapunov and Hurwitz stability analyses. We provide all the detailed insights and proofs in the supplementary material.

5 Identifiability of the games

We now characterize the conditions under which LAGs become identifiable in terms of strategic interactions. Our recovery procedure is a novel adaptation of the primal-dual witness method [44] to games. Specifically, we estimate from data $D$ the support $S_{i}$ or the set of neighbors of $i$ defined wrt to the unknown influences $w_{ij}^{*}\neq 0$. We also recover the correct sign of these influences. We focus on the one-shot setting (i.e., $k=1$) with the gradient update so that the observed outcome is sampled from player strategies after one round of communication. We also use binary actions to simplify the exposition. We denote by $\phi_{j}^{(m)}$ the probability assigned by the initial strategy $\sigma_{j}^{0}(x^{(m)})$ to action $1$ for player $j$ on example $m$. Let $\ell_{i}(w_{i};D)$ be the average cross-entropy loss between one-step strategies under candidate weights $w_{i}\triangleq(w_{ij})_{j\neq i}$ and the observed actions for player $i$, and $\lambda>0$ be a regularization parameter. We solve the following problem for each player $i\in[n]$:

[TABLE]

Let $H_{i}^{M}$ be the sample Hessian $\nabla^{2}\ell_{i}(w_{i};D)$ under $w_{i}$, and $H_{i}^{*M}$ pertain to true weights $w_{i}^{*}$. Let $\Lambda_{min}(\cdot)$ and $\Lambda_{max}(\cdot)$ denote the minimum and the maximum eigenvalues. The following assumptions serve as our analogues of the conditions for support recovery in Lasso [44] and Ising models [45]:

[TABLE]

where $C_{\min}>0$, $C_{\max}<\infty$, $\gamma\in(0,1]$; $|||A|||_{\infty}$ is the maximum over $L_{1}$-norm of rows in $A$; $H_{i,SS}^{*M}$ is the submatrix obtained by restricting $H_{i}^{*M}$ to rows and columns corresponding to neighbors, i.e., players in $S_{i}$, and $H_{i,SS^{c}}^{*M}$ is restricted to rows pertaining to $S_{i}$ and columns to $S^{c}_{i}$ (non-neighbors). Let the number of neighbors for any player be at most $d\leq n-1$. We have the following result.

Theorem 3**.**

*Let $M>\dfrac{80^{2}C_{\max}^{2}}{C_{\min}^{4}}\left(\dfrac{2-\gamma}{\gamma}\right)^{4}d^{2}\log(n)$, and $\lambda\geq\dfrac{8\alpha(2-\gamma)}{\gamma}\sqrt{\dfrac{\log(n)}{M}}$. Suppose the data satisfies assumptions (11), and (12). The following results hold with high probability for each $i\in[n]$: (a) the corresponding optimization problem (10) has a unique solution, i.e., a unique set of neighbors for $i$, and (b) the set of predicted neighbors of $i$ is a subset of the true neighbors. Additionally, the predicted set contains all true neighbors $j$ for which $|w_{ij}^{*}|\geq 10\sqrt{d}\lambda/(\alpha^{2}C_{\min})$. *

6 Experiments

We now describe the results of our experiments that provide insights into some important aspects of our games. We first show that LAG qualitatively recovers the known strategic behavior of the Justices in the longest serving US Supreme Court. We also provide quantitative evidence that LAG outperforms the prior methods on two different measures. We then demonstrate that the structure estimated by LAG on the UN General Assembly data [46] is meaningful, and helps unravel the subtle behavior of member countries. Finally, we present evidence to underscore that LAGs can be effectively transferred to predict strategies in new settings with different sets of players.

We found that LAGs performed well over a wide range of hyperparameters. We implemented the models in sections 2 and 3 with $L_{1}$ regularization for structure estimation as described in section 5. Our models performed well for a wide range of $\alpha$, $\lambda$, and $k$. We report the results with $k=5$, $\alpha=0.1$, and $\lambda=0.1$ for all our experiments, except the transferable setting where we set $\alpha=0.01$ and $\lambda=0$.111We did not impose $L_{1}$ penalty in the transferable setting since the interactions are learned for new individuals, i.e., they are not specific to any fixed set of players, unlike the basic setting. We set $\nu$ to be the identity function in (4). We trained our models in batches of size 200, with default settings of the RMSprop optimizer in PyTorch. To account for effect of randomness in neural training toward structure estimation, we averaged the parameters of each model across 5 independent runs. We ordered the influence of neighbors based on the corresponding estimated average weights from the most positive to the most negative.

6.1 US Supreme Court Data

We included all the cases from the Rehnquist Court, during the period 1994-2005 that had votes documented for all the 9 Justices (i.e. our players). Justices Rehnquist (R), Scalia (Sc), and Thomas (T) represented the conservative side; Justices Stevens (St), Souter (So), Ginsburg (G), and Breyer (B) formed the liberal bloc; and Justices Kennedy (K) and O’Connor (O), often called swing votes, followed a moderate ideology. Our contexts comprise of 32 binary attributes that characterize the specifics of the appeal, e.g., the disposition of lower court. Our observation outcome $y^{(m)}$ for each context $x^{(m)}$ pertains to the corresponding votes of the Justices. The votes belong to one of the three categories: yes, no, or complex.

Fig. 1 describes in detail how our method yields a structure that is qualitatively consistent with the known jurisprudence of the Rehnquist Court. Specifically, (a) the conservatives and the liberals form two separate coherent, strongly-connected blocs that are well-segregated from each other, and (b) they influence the moderates but not vice-versa.

Fig. 2 provides a detailed quantitative comparison that reveals how LAG compared favorably with both the prior PSNE based methods [12, 13] on two separate measures: (a) recovering edges consistent with the ideology of the Justices, and (b) coherence in terms of size of the cut, i.e., minimum number of edges to be removed in order to decompose the recovered structure into the constituent blocs.

6.2 United Nations General Assembly (UNGA) Data

Our second dataset consists of the roll call votes of the member countries on the resolutions considered in the UN General Assembly. Each resolution is a textual description that provides a context while the votes of the countries on the resolution pertain to the observed outcome. We compiled data on all resolutions in UNGA from 1992 onward to understand the interactions of member nations since the dissolution of the Soviet Union. We considered 25 countries that have dominated the United States (USA) politics, and are generally known to belong to one of the two blocs: pro-USA, namely, Australia (AUS), Canada (CAN), France (FRA), Germany (GER), Israel (ISR), Italy (ITA), Japan (JPN), South Korea (KOR), Norway (NOR), Ukraine (UKR); and others, namely, Afghanistan (AFG), Belarus (BLR), China (CHN), Cuba (CUB), Iran (IRN), Iraq (IRQ), Mexico (MEX), Pakistan (PAK), Philippines (PHL), North Korea (PRK), Kazakhstan (KAZ), Russia (RUS), Syria (SYR), Venezuela (VEN) and Vietnam (VNM). We used pretrained GLoVe embeddings to represent each resolution as a 50-dimensional context vector $x^{(m)}$. Each vote was interpreted to take one of the three values: 1 (yes), 2 (absent/abstain), or 3 (no), and we represented $y^{(m)}$ as a 26-dimensional vector.

Fig. 3 shows the structure estimated by our method, i.e., the weights $\hat{w}_{i}$ learned for each country $i$. The weights $\hat{w}_{ij},j\neq i$ have a natural interpretation in terms of influence: the more positive $w_{ij}$ is, the more positive the influence of $j$ on $i$. A similar connotation holds for the negative weights. To aid visualization, we disentangle the positive and the negative connections, and depict only the most influential connections for either case. Fig. 3 describes how our method estimated a meaningful structure, and unraveled subtle influences beyond the prominent two-bloc structure. The learned type parameters $\hat{\theta}_{i}$ were also found to be similar for members in the same bloc (Fig. 4).

Our final set of experiments focused on transferring LAGs. Besides UNGA data, we compiled a 73-dimensional feature vector for each country from its HDI Indicators [47]. We then kept aside three-fourths of the data to set up games over small sets of players by randomly sampling 5 countries independently for each context. The left-out players from these contexts were then sampled independently to get another set of 5 countries per context. We call these two sets A and B. We formed a third set C of 5 countries per game from the untouched data (i.e. the one-fourth fraction). Thus, A and B were defined on the same contexts, that were disjoint from C. We averaged results over 10 such independent triplets (A, B, C) to mitigate sampling effects. We trained a model for A using the procedure in section 3, and computed the loss on B. Our baseline, train empirical, used the empirical distribution of actions for each player $i$ over the games it participated in A as its predicted strategy for the games in B and C. The loss of LAGs (0.852) turned out to be lower than the baseline (0.865) on C. Moreover, as Fig. 4 shows, the loss of LAGs (0.696) was found to be significantly lower than the baseline (0.720) on B even though HDI does not fully reflect complex country characteristics. Thus, our results clearly underscore the benefits of using LAGs for strategic prediction.

Appendix A Convergence of dynamics

We first provide some insight into our proof techniques. We prove convergence to SNE via carefully crafted Lyapunov functions $\mathcal{V}$ that are locally positive definite and have a locally negative semidefinite time derivative, and thus satisfy the Lyapunov stability criterion. The other proofs track the evolution of game dynamics around an equilibrium, where $\dot{q}_{i}=0$ and $\dot{r}_{i}=0$. Specifically, we analyze conditions under which the Jacobian matrix of the linearization is Hurwitz stable, i.e., all the eigenvalues have negative real roots, and exploit the fact that the behavior of the ODE near equilibrium is same as its linear approximation when the real parts of all eigenvalues are non-zero. Our discrete updates would then converge to a Nash equilibrium with positive probability [43].

Recall that AA reveals more information about the evolution of neighbors’ strategy. As a result, the PA settings, i.e. (7) and (9), require additional subtle reasoning since at equilibrium $r_{i}^{*}$ converges only to $\mathcal{A}_{i}(q_{-i}^{*})$ and not to $q_{i}^{*}$. Since $q_{i}$ evolves within $\Delta(A)$, stochasticity assumptions are required to ensure $r_{i}$ stays within the probability simplex as well. Note that the LAG-FP updates to strategies are smooth due to the entropy term (since $\tau>0$), unlike LAG-GP. Consequently, the results for LAG-GP require a separate treatment of completely mixed NE and strict NE, unlike LAG-FP where they can be analyzed without distinction. Note that $\tau>0$ ensures that best response is a singleton set and therefore we could leverage the ODE formulations. Differential inclusions [30, 31] could be used instead to handle $\tau=0$. Our stability conditions can be simplified further when $\lambda$ is sufficiently large, whence the behavior may be understood solely in terms of $\gamma$. In general, via standard eigenvalue arguments [43], our protocols admit stable linearization under mild conditions.

We now provide detailed proofs on convergence of dynamics. We use AA1, AA2 etc. to indicate that the result pertains to convergence in an active aggregator setting. Likewise, we will use PA1 etc. for the passive aggregator setting. We start with the active aggregator.

Theorem AA1**.**

(Convergence under LAG-FP/AA to NE)* Let $(q_{1}^{*},\ldots,q_{n}^{*},z_{1},\ldots,z_{n})$ be a NE under the dynamics in (6). There exists a matrix $\mathcal{D}$ such that the linearization of (6) with $\gamma>0$ is locally asymptotically stable for $\lambda>0$ if and only if the following matrix is stable*

[TABLE]

Proof.

Since $\tau>0$, best response is a singleton set, and the unique best response $\sigma_{i}^{*}$ can be obtained by setting the gradients of the payoff functions to 0. In particular, we have the best response

[TABLE]

where $\zeta$ is the softmax function with output coordinate $\ell$ given by

[TABLE]

Now recall from (6) that we have the following ODE:

[TABLE]

Since $\beta_{i}^{\tau}$maps it input to the simplex $\Delta(A)$, we note that the right side of (14) is a difference between two probability distributions. Therefore this difference must sum to zero. Moreover, since $|A|=m$, we have $m-1$ degrees of freedom that can be used to express this difference. Therefore, we can investigate the evolution of $q_{i}$ via a matrix $N\in\mathbb{R}^{m\times(m-1)}$ of $(m-1)$ orthonormal columns such that

[TABLE]

where $I_{m-1}$ is the identity matrix of order $m-1$, and $1_{m}$ and $0_{m}$ are $m$-dimensional vectors with all coordinates set to 1 and 0 respectively. We will sometimes omit the subscripts for $I_{m}$, $1_{m}$, and $0_{m}$ when the size will be clear from the context. The equilibrium $(q_{i}^{*},q_{-i}^{*})$ corresponds to a point $(q_{i}(t)=q_{i}^{*},q_{-i}(t)=q_{-i}^{*},r_{i}(t)=q_{i}^{*},r_{-i}(t)=q_{-i}^{*})$ of the dynamics. It will be convenient to investigate the dynamics as the evolution of deviations around this point. Since $q_{i}$ is confined to $\Delta(A)$, we can express

[TABLE]

where $\delta x_{q_{i}}(t)\in\mathbb{R}^{m-1}$ is uniquely specified, and likewise $r_{i}=q_{i}^{*}+\delta x_{r_{i}}(t)$ for some $\delta x_{r_{i}}(t)$. Thus, we can define a block diagonal matrix $\mathcal{N}\in\mathbb{R}^{2nm\times 2n(m-1)}$, with each diagonal block set to $N$ and all other elements set to 0, such that

[TABLE]

where

[TABLE]

is formed by stacking together the deviations at time $t$ in a column vector. Then, the following is immediate from (16):

[TABLE]

Denote the Jacobian matrix obtained by taking derivatives of vector $y$ with respect to vector $x$ by $J_{x}y$. We will linearize $\dot{q}_{i}=F_{i}(q_{i},q_{-i},r_{-i})$ in (14) around $\triangleq(q_{1}^{*},q_{-1}^{*},q_{1}^{*},q_{-1}^{*})$ using first order Taylor series. Then, since $\dot{q}_{i}^{*}=0$, we note from (14) and (17) that

[TABLE]

Now, at equilibrium, we have $\dot{q}_{i}=0$ for all $i\in[n]$, and therefore we have from (14) that

[TABLE]

Let ${\rm diag}(b)$ be a diagonal matrix with vector $b$ on the diagonal and all other elements set to 0. Ignoring the second order and higher terms, we therefore have by the Taylor series approximation that

$\hskip 85.35826ptF_{i}(q_{i},q_{-i},r_{-i})$

[TABLE]

where $\tilde{D}_{ik}\triangleq\dfrac{w_{ik}}{\tau}\nabla\zeta\left(\dfrac{A_{i}(q_{-i}^{*})-z_{i}}{\tau}\right)$, and $\nabla\zeta(b)\triangleq{\rm diag}(\zeta(b))-\zeta(b)\zeta^{\top}(b)\leavevmode\nobreak\ .$

Define $D_{ik}=N^{\top}\tilde{D}_{ik}N$. Since $N^{\top}N=I_{m-1}$, it follows immediately from (18) that

[TABLE]

Linearizing (15), we see that the Taylor approximation results in

[TABLE]

We define

[TABLE]

Combining (19) and (20) together, we can write

[TABLE]

The statement of the theorem now follows immediately from the Hurwitz stability criterion. ∎

Theorem AA2**.**

(Convergence under LAG-GP/AA to CMNE)* Let $(q_{1}^{*},\ldots,q_{n}^{*},z_{1},\ldots,z_{n})$ be a completely mixed NE under the dynamics in (8). Then the linearization of (8) with $\gamma>0$ is locally asymptotically stable for $\lambda>0$ if and only if the following matrix is stable*

[TABLE]

Proof.

Recall the ODE from (8):

[TABLE]

At equilibrium $(q_{1}^{*},\ldots,q_{n}^{*},z_{1},\ldots,z_{n}),\dot{q}_{i}=0$ and $\dot{r}_{i}=0$. Therefore, using (26), we have:

[TABLE]

Since the equilibrium is completely mixed, $q_{i}^{*}$ is in the interior of $\Delta(A)$. We invoke Lemma 4.1 in [43] to get the following:

[TABLE]

for $\delta y$ sufficiently small, and $N$ as defined in the proof of Theorem AA1. Then, for a sufficiently small deviation $\delta x$, where $\delta x$ is as defined in Theorem 1, we get the following dynamics:

[TABLE]

Linearizing these equations and noting that $N^{\top}N=I$, we get

[TABLE]

and

[TABLE]

It follows immediately that

[TABLE]

where the weight matrix

[TABLE]

∎

Theorem AA3**.**

(Convergence under LAG-GP/AA to SNE)* Let $(q_{1}^{*},\ldots,q_{n}^{*},z_{1},\ldots,z_{n})$ be a strict NE under the dynamics in (8). The associated equilibrium point $(q_{i}=q_{i}^{*},q_{-i}=q_{-i}^{*},r_{i}=q_{i}^{*},r_{-i}=q_{-i}^{*})$ is locally asymptotically stable for any $\gamma>0$ and $\lambda>0$.*

Proof.

Recall the ODE from (8):

[TABLE]

To prove the local asymptotic stability of the ODE dynamics, we will define a Lyapunov function $\mathcal{V}$ that is locally positive definite and has locally negative semi-definite time derivative. Consider

[TABLE]

We define the shorthand $d_{i}\triangleq q_{i}+\mathcal{A}_{i}(q_{-i}+\gamma\dot{r}_{-i})-z_{i}$. Applying the chain rule, we see that the time derivative of $\mathcal{V}$,

[TABLE]

Also, we note that

[TABLE]

This immediately implies

[TABLE]

Consider $(B)=\left(\Pi_{\Delta}(d_{i})-q_{i}^{*}\right)^{\top}\left(\Pi_{\Delta}(d_{i})-q_{i}\right)$. Since $\Delta(A)$ is a convex set, the projection property implies

[TABLE]

whence we note

[TABLE]

Now, we note from the definition of $\mathcal{V}$ in (A) that by decreasing the distances $(q_{i}-r_{i})$ and $(q_{i}-q_{i}^{*})$, we can make $\mathcal{V}(q_{i},q_{-i},r_{i},r_{-i})$ arbitrarily close to 0 from the right. In other words, we can consider a sufficiently small neighborhood around the equilibrium such that as $r_{i},q_{i}\to q_{i}^{*}$, (B) tends to

[TABLE]

for some sufficiently small $\delta y$ and $\Pi_{\Delta}(d_{i})\neq q_{i}^{*}$. The last inequality follows since $(q_{1}^{*},\ldots,q_{n}^{*},z_{1},\ldots,z_{n})$ is a strict Nash equilibrium, whereby (a) $(q_{i}^{*},q_{-i}^{*})$ is a pure strategy Nash equilibrium (since $\mathcal{A}_{i}(\cdot)$ is a linear transformation and the payoff maximization happens at the vertex), the and (b) $q_{i}^{*}$ is a (strictly) best response to $q_{-i}^{*}$ and nearby strategies. Therefore, we see from (A) that for a sufficiently small neighborhood around the equilibrium point,

[TABLE]

where we have invoked the Cauchy-Schwarz inequality in the penultimate line. Since this quantity is non-positive, we see that $\dot{\mathcal{V}}$ is locally negative semi-definite. Finally, it is clear from (A) that $\mathcal{V}(q_{i},q_{-i},r_{i},r_{-i})>0$ in the neighborhood $(q_{i},q_{-i},r_{i},r_{-i})$ of the equilibrium point $(q_{i}=q_{i}^{*},q_{-i}=q_{-i}^{*},r_{i}=q_{i}^{*},r_{-i}=q_{-i}^{*})$, and $\mathcal{V}(q_{i}^{*},q_{-i}^{*},q_{i}^{*},q_{-i}^{*})=0$. Thus, $\mathcal{V}$ is locally positive definite, and the statement of the theorem follows. ∎

We will now characterize conditions for convergence in the passive aggregator setting.

Theorem PA1**.**

(Convergence under LAG-FP/PA to NE)* Let the weight matrix $W$ be stochastic. Let $(q_{1}^{*},\ldots,q_{n}^{*},z_{1},\ldots,z_{n})$ be a NE under the dynamics in (7). There exists a matrix $\mathcal{D}_{1}$ with zero diagonal, and a block diagonal matrix $\mathcal{D}_{2}$ such that the linearization of (7) with $\gamma>0$ is locally asymptotically stable for $\lambda>0$ if and only if if the following matrix is stable*

[TABLE]

Proof.

We reproduce the ODE from (7):

[TABLE]

Note that at equilibrium $\dot{r}_{i}=0$, but unlike Theorem AA1, $r_{i}$ does not converge to $q_{i}^{*}$. Specifically, we note that the equilibrium $(q_{i}^{*},q_{-i}^{*})$ corresponds to a point $(q_{i}(t)=q_{i}^{*},q_{-i}(t)=q_{-i}^{*},r_{i}(t)=\mathcal{A}_{i}(q_{-i}^{*})),i\in[n]$, of the dynamics. Therefore, we will instead linearize around this point. Since the weight matrix $W$ is stochastic, we must have $\mathcal{A}_{i}(q_{-i}^{*})\in\Delta(A)$. Therefore, we can investigate the deviation of $r_{i}$ around $\mathcal{A}_{i}(q_{-i}^{*})$ with the help of matrix $N$ defined in Theorem AA1. In particular, we can express the deviation vector $\delta x=(\delta x_{q_{1}},\ldots,\delta x_{q_{n}},\delta x_{r_{1}},\ldots,\delta x_{r_{n}})^{\top}$ as:

[TABLE]

where the block diagonal matrix $\mathcal{N}$ is as defined in Theorem AA1. Linearizing around our equilibrium point and proceeding similarly to Theorem AA1, we get

[TABLE]

where

[TABLE]

[TABLE]

and

[TABLE]

with $\zeta(b)$ the same as in Theorem AA1. Additionally, we have

[TABLE]

Recall that the weight matrix

[TABLE]

Define

[TABLE]

[TABLE]

Then, the proof follows by combining (32) and (33), since we can express the deviations as

[TABLE]

∎

Theorem PA2**.**

(Convergence under LAG-GP/PA to CMNE)* Let the weight matrix $W$ be stochastic. Let $(q_{1}^{*},\ldots,q_{n}^{*},z_{1},\ldots,z_{n})$ be a completely mixed NE under the dynamics in (9). Then the linearization of (9) with $\gamma>0$ is locally asymptotically stable for $\lambda>0$ if and only if the following matrix is stable*

[TABLE]

Proof.

Recall the ODE from (9):

[TABLE]

At equilibrium $(q_{1}^{*},\ldots,q_{n}^{*},z_{1},\ldots,z_{n}),\dot{q}_{i}=0$ and $\dot{r}_{i}=0$. Therefore, using (34), we have:

[TABLE]

Proceeding along the lines of proof of Theorem AA2, for a sufficiently small deviation $\delta x$ as defined in Theorem PA1, we can equivalently analyze the following dynamics:

[TABLE]

Linearizing these equations and noting $N^{\top}N=I$, we get

[TABLE]

and

[TABLE]

It follows immediately that

[TABLE]

where the weight matrix

[TABLE]

∎

Theorem PA3**.**

(Convergence under LAG-GP/PA to SNE)* Let the weight matrix $W$ be doubly stochastic. Let $(q_{1}^{*},\ldots,q_{n}^{*},z_{1},\ldots,z_{n})$ be a strict NE under the dynamics in (9). The associated equilibrium point $(q_{i}=q_{i}^{*},r_{i}=A_{i}(q_{-i}^{*}))_{i\in[n]}$ is locally asymptotically stable for sufficiently small $\gamma\lambda$, where $\gamma>0$ and $\lambda>0$.*

Proof.

Recall the ODE from (9):

[TABLE]

We will prove local asymptotic stability via a Lyapunov function $\mathcal{V}$ that is locally positive definite and has locally negative semi-definite time derivative. Consider

[TABLE]

We define the shorthand $\tilde{d}_{i}\triangleq q_{i}+\mathcal{A}_{i}(q_{-i})+\gamma\dot{r}_{-i}-z_{i}$. Applying the chain rule, we see that the time derivative of $\mathcal{V}$,

[TABLE]

Also, we note that

[TABLE]

Proceeding along the lines of Theorem AA3, for sufficiently small $\gamma\lambda$ and sufficiently small neighborhood around the equilibrium point, we can show that

[TABLE]

by noting that $w_{ki}\geq 0$ for all $i\in[n]$, $k\neq i$ and invoking Cauchy-Schwarz. Furthermore, since $W$ is doubly stochastic, we have $\sum_{k\neq i}w_{ki}=1$ and $\sum_{k\neq i}w_{ik}=1$ for all $i\in[n]$. Thus, we may decompose the second term on the right in the last equation as

[TABLE]

The first term in the last equation may be interpreted as a weighted outgoing flow from player $i$ to player $k\neq i$. Now viewing this from the dual perspective of the total incoming flow, we equivalently have

[TABLE]

which implies that $\dot{\mathcal{V}}$ is locally negative semi-definite. The local positive definiteness of $\mathcal{V}$ may be argued along the same lines as the proof of Theorem AA3 and we are done. ∎

Appendix B Identifiability of one-shot LAGs

We now present the results on provably recovering the structure of one-shot LAGs from data. Specifically, we characterize the conditions under which LAGs with one step dynamics become identifiable, and provide an algorithm to recover the structure of the game, i.e., the neighbors for each player $i\in[n]$ with the signs (positive or negative) of their respective influences.

Our recovery procedure adapts the primal-dual witness method [44] for structure estimation in games. The method has previously been applied in several non-strategic settings such as Lasso [44] and Ising models [45]. Recently, [34] employed this method to recover a set of pure strategy Nash equilibria (PSNE) from data consisting of a subset of PSNE, and a small fraction of non-equilibrium outcomes assumed to be sampled under their noise models in the setting of linear influence games. However, the problem of structure recovery is significantly harder: it is known [14, 34] that the problem becomes non-identifiable in the setting of PSNE, since multiple game structures may pertain to the same of PSNE. To our knowledge, there are no known results on the provable recovery of structure of graphical games from data. We fill this gap by characterizing conditions under which one-shot LAGs become identifiable.

Our approach follows the general proof structure of primal-dual witness method in the context of model selection for Ising models [45]. However, our setting is significantly different from the setting in [45] where context and dynamics play no part, and all the observed data is assumed to be sampled from a common (global) distribution expressible in a closed form. In contrast, each observed outcome in our setting is sampled from a separate joint strategy profile following one-step of dynamics initiated under a different context.

Specifically, in the one-shot setting, consider a dataset $D=\{(x^{(m)},a^{(m)})\in\mathcal{X}\times\mathcal{Y},m\in[M]\}$ where $a^{(m)}$ is the action profile (i.e. observed outcome) sampled from the joint player strategies after one round of communication. Assume that the type parameters $\theta=(\theta_{1},\ldots,\theta_{n})$ are known. Then, since types for any context are determined by the parameters $\theta$, we have access to the player types $z^{(m)}(x^{(m)})=(z_{1}^{(m)},\ldots,z_{n}^{(m)})$, which in turn determine determine the initial strategies for all the examples $m\in[M]$. We focus on binary actions here since they let us simplify the exposition while conveying the essential ideas. Specifically, each player $i\in[n]$ initially plays action 1 with probability

[TABLE]

and the action 0 with probability $1-\phi_{i}^{(m)}$. We define $\phi^{(m)}=(\phi_{1}^{(m)},\ldots,\phi_{n}^{(m)})$, and $\Phi_{-i}^{(m)}\leavevmode\nobreak\ \leavevmode\nobreak\ =\leavevmode\nobreak\ \leavevmode\nobreak\ (\phi_{j}^{(m)})_{j\neq i}$. We focus on the gradient update setting where after one round of communication, player $i$ responds to its neighbors with its updated strategy $(\sigma_{i}^{*(m)},1-\sigma_{i}^{*(m)})$, where

[TABLE]

such that $\alpha>0$, and $w_{ij}^{*}\in\mathbb{R}$ is the true influence (i.e. interaction weight) of player $j\in[n]\setminus\{i\}$ on $i$. Recall that we call player $j$ a neighbor of $i$ if $|w_{ij}^{*}|>0$. Finally, action $a_{i}^{(m)}$ is sampled from the updated strategy, and we obtain the joint profile $a^{(m)}=\{a_{i}^{(m)},i\in[n]\}$ as the observed outcome. Our goal is to estimate, from $D$ and $\alpha$, the support $S_{i}$, or the set of neighbors $j$ for $i$, i.e., the players that have influence $w_{ij}^{*}\neq 0$. We can thus separate the influence of neighbors of $i$ from the non-neighbors by defining the set of non-zero weights $w_{i,S}^{*}=\{w_{ij}^{*}|j\in S_{i}\}$. We denote the complement of a set $A$ by $A^{c}$. Thus, $w_{ij}^{*}=0$ for $j\in S_{i}^{c}$. We equivalently write $w_{i,S^{c}}^{*}=\boldsymbol{0}$. We are interested in recovering not only the support of each player $i$, but also the correct sign of influence (i.e. positive or negative) of each neighbor $j$ on $i$.

We consider the average cross-entropy loss between the strategy under $w_{i}$ and the observed outcome.

[TABLE]

We compute the gradient and the Hessian of the sample loss:

[TABLE]

[TABLE]

We will often use the variance function $\eta_{i}(w_{i};m)\triangleq\alpha^{2}\sigma_{i}^{(m)}(1-\sigma_{i}^{(m)})$ as a shorthand, and write

[TABLE]

We denote by $H_{i,SS}^{*M}$ the submatrix obtained by restricting the Hessian $H_{i}^{*M}$, pertaining to true weights, to rows and columns corresponding to neighbors, i.e., players in $S_{i}$. Likewise, $H_{i,SS^{c}}^{*M}$ denotes the submatrix restricted to rows pertaining to $S_{i}$ (neighbors) and columns to $S^{c}_{i}$ (non-neighbors).

We will provide detailed analysis under sample Fisher matrix assumptions. We will omit the analysis for the population setting that can be derived by imposing analogous assumptions directly on the population matrices, and making concentration arguments that show these assumptions hold in the sampled setting with high probability. Recall from the main text that we make the following assumptions that are reminiscent of those for support recovery under Lasso [44], and model selection in Ising models [45]. We first recall our assumptions from the main text.

Assumptions.

[TABLE]

[TABLE]

[TABLE]

such that $C_{\min}>0$, $C_{\max}<\infty$, and $\gamma\in(0,1]$. In our notation, $|||A|||_{\infty}$ denotes the maximum $\ell_{1}$ norm across rows of matrix $A$, and $|||A|||_{2}$ denotes the spectral norm (i.e. maximum singular value) of $A$. $\Lambda_{min}(A)$ and $\Lambda_{max}(A)$ refer respectively to the minimum and the maximum eigenvalue of a square matrix $A$.

Analysis. We propose to solve the following regularized problem for each player $i\in[n]$ separately.

[TABLE]

where $\lambda_{M,n,d}>0$ is a regularization parameter that depends on the sample size $M$, the number of players $n$, and the maximum degree (i.e. number of neighbors) of any player. For brevity, we will omit the dependence of this parameter on $n$ and $d$, and simply write $\lambda_{M}$. This problem is convex but not differentiable everywhere because of the $\ell_{1}$ penalty. Note that since the problem is not strictly convex, it might have multiple minimizing solutions. For any such optimal solution $\hat{w}_{i}$, we must have by KKT conditions,

[TABLE]

where the subgradient $\hat{\kappa}_{i}\in\mathbb{R}^{n-1}$ is such that

[TABLE]

We would like to ensure the following conditions in order to recover the signed neighborhood for $i$.

[TABLE]

Our analysis is built on the primal-dual witness (PDW) method [44]. This method has the following steps. First, only for the sake of analysis, we presuppose that some Oracle provides the true neighbors $S_{i}$. Therefore, we solve the following problem to recover the signs of true neighbors.

[TABLE]

We then set the components of the dual vector $\kappa_{i}$ that pertain to neighbors of $i$ to the sign of corresponding components in $\hat{w}_{i,S}$. That is, $\hat{\kappa}_{i,j}\leavevmode\nobreak\ =\leavevmode\nobreak\ \text{sign}(\hat{w}_{i,j}),\,\forall j\in S_{i}$. We next set $\hat{w}_{i,S^{c}}=\boldsymbol{0}$, and thus (48) is satisfied. We then solve for $\hat{\kappa}_{i,S^{c}}$ by plugging $\hat{w}_{i,S}$, $\hat{\kappa}_{i,S}$, and $\hat{w}_{i,S^{c}}$ in (45). Thus, we are left to show that (46) and (47) are satisfied. We impose conditions on $M$, $n$, and $d$ under which these conditions are satisfied with high probability. In fact, we prove a stronger result for (46), namely, strict dual feasibility for non-neighbors, i.e., $|\hat{\kappa}_{i,j}|<1$ for all $j\in S_{i}^{c}$.

We argue that our construction yields a unique optimal primal solution $\hat{w}_{i}$. Specifically, we invoke Lemma 1 from [45] that states that so long as $||\hat{\kappa}_{i,S^{c}}||_{\infty}<1$, any optimal primal solution $\tilde{w}_{i}$ satisfies $\tilde{w}_{i,S^{c}}=\boldsymbol{0}$. This is established by our construction above. Moreover, this Lemma asserts that $\hat{w}_{i}$ is the unique solution to (44) if $\Lambda_{\min}(\hat{H}_{i,SS}^{M})>0$, i.e., if the sample Hessian under $\hat{w}_{i}$ is positive definite when restricted to the rows and columns in the true support $S_{i}$. We show that assumption (41) implies $\Lambda_{\min}\left(\hat{H}_{i,SS}^{M}\right)\leavevmode\nobreak\ \geq\leavevmode\nobreak\ \alpha^{2}\dfrac{C_{min}}{2}>0$, and this guarantees that we correctly recover the signed neighborhood of $i$.

To proceed, we define $G_{i}^{M}=-\nabla\ell_{i}(w_{i}^{*};D)$ and rewrite (45) as

[TABLE]

Applying the mean value theorem component-wise, we can write (50) as

[TABLE]

where

[TABLE]

for some vector $\overline{w}_{i}^{(j)}=t_{j}\hat{w}_{i}+(1-t_{j})w_{i}^{*}$, $t_{j}\in[0,1]$. Here, $(A)^{\top}_{j}$ denotes row $j$ of matrix $A$.

We are now ready to state an important lemma. We will use R1, R2 etc. to indicate that the result is aimed toward provable recovery (i.e. identifiability) of our games.

Lemma R1**.**

We have that

[TABLE]

which converges to zero at rate $\exp(-C_{\alpha,\lambda}\lambda_{M}^{2}M)$ (where constant $C_{\alpha,\gamma}$ depends on $\alpha$ and $\gamma$) whenever

[TABLE]

Proof.

We note that

[TABLE]

where $|Z_{i,m}^{u}|\leq\alpha$ for each component $Z_{i,u}^{m}$ of random vector $Z_{i}^{m}$. Moreover, $\mathbb{E}(Z_{i,u}^{m})=0$ under $w_{i}^{*}$, and $Z_{i,u}^{1},\ldots,Z_{i,u}^{M}$ are independent. Invoking the Hoeffding’s inequality, we have that for any $\delta>0$,

[TABLE]

where $G_{i,u}^{M}$ denotes the component at index $u$ of vector $G_{i}^{M}$. Setting $\delta=\dfrac{\gamma\lambda_{M}}{4(2-\gamma)}$, we get

[TABLE]

Then, applying a union bound over indices $u\in[n-1]$, we get

[TABLE]

∎

Lemma R2**.**

Let $\lambda_{M}d\leavevmode\nobreak\ \leq\leavevmode\nobreak\ \dfrac{\alpha C^{2}_{\min}}{10C_{\max}}$ and $||G_{i}^{M}||_{\infty}\leavevmode\nobreak\ \leq\leavevmode\nobreak\ \dfrac{\lambda_{M}}{4}$. Then,

[TABLE]

Proof.

We define a function $F:\mathbb{R}^{d}\to\mathbb{R}$ that quantifies the change in optimization objective at a distance $\Delta_{i,S}$ from the true parameters $w^{*}_{i,S}$. Specifically,

[TABLE]

Note that $F$ is convex and $F(\boldsymbol{0})\leavevmode\nobreak\ =\leavevmode\nobreak\ 0$. Moreover, $F$ is minimized for $\hat{\Delta}_{i,S}=\hat{w}_{i,S}-w^{*}_{i,S}$. Therefore, $F(\hat{\Delta}_{i,S})\leq 0$. We show that the function $F$ is strictly positive on the surface of a Euclidean ball of radius $B$ for some $B>0$. Then, the vector $\hat{\Delta}_{i,S}$ lies inside the ball, i.e.,

[TABLE]

This follows since otherwise, the convex combination $t\hat{\Delta}_{i,S}+(1-t)\boldsymbol{0}$ would lie on boundary of the ball for some $t\in(0,1)$, which would imply the contradiction

[TABLE]

Therefore, let $\Delta\in\mathbb{R}^{d}$ be an arbitrary vector such that $||\Delta||_{2}=B$. We then have from Taylor’s series

[TABLE]

for some $\theta\in[0,1]$. We lower bound $F(\Delta)$ by bounding each term on the right side of (52).

We let $B=O\lambda_{M}\sqrt{d}$ where we will choose $O>0$ later. From Cauchy-Schwartz inequality,

[TABLE]

where in the last inequality we have used $||\Delta||_{2}=B=O\lambda_{M}\sqrt{d}$, and

[TABLE]

by our assumption on $||G_{i}^{M}||_{\infty}$ in the lemma statement. Next, by triangle inequality, we have

[TABLE]

We now bound the quantity $\Delta^{\top}\nabla^{2}\ell(w_{i,S}^{*}+\theta\Delta;D)\Delta$. We note that

[TABLE]

Applying Taylor’s series expansion, we note that $\Delta^{\top}\nabla^{2}\ell(w_{i,S}^{*}+\theta\Delta;D)\Delta$

[TABLE]

Now, a simple calculation shows that $|\eta_{i}^{\prime}(\cdot)|\leavevmode\nobreak\ \leq\leavevmode\nobreak\ \alpha^{3}$. Moreover, we note for $\tilde{\theta}\in[0,1]$,

[TABLE]

Putting all these facts together, along with our assumption (42), we get

[TABLE]

when $\lambda_{M}\leq\dfrac{C_{\min}}{2\alpha C_{\max}Od}$ . Therefore, plugging the lower bounds from (53), (56), and (57) in (52),

[TABLE]

for $O=\dfrac{5}{\alpha^{2}C_{\min}}$. Thus, for $\lambda_{M}\leavevmode\nobreak\ \leq\leavevmode\nobreak\ \dfrac{C_{\min}}{2\alpha C_{\max}Od}\leavevmode\nobreak\ =\leavevmode\nobreak\ \dfrac{\alpha C^{2}_{\min}}{10C_{\max}d}$, we must have

[TABLE]

∎

Lemma R3**.**

Let $\lambda_{M}d\leavevmode\nobreak\ \leq\leavevmode\nobreak\ \dfrac{\alpha C_{\min}^{2}}{100C_{\max}}\dfrac{\gamma}{2-\gamma}$ and $||G_{i}^{M}||_{\infty}\leavevmode\nobreak\ \leq\leavevmode\nobreak\ \dfrac{\lambda_{M}}{4}$. Then,

[TABLE]

Proof.

We have for $j\in[n]\setminus\{i\}$ and some $\overline{w}_{i}^{(j)}=t_{j}\hat{w}_{i}+(1-t_{j})w_{i}^{*}$, $t_{j}\in[0,1]$,

[TABLE]

where $\overline{\overline{w}}_{i}^{(j)}$ is a point on the line between $\overline{w}_{i}^{(j)}$ and $w_{i}^{*}$, by the mean value theorem. We note that

[TABLE]

We thus write

[TABLE]

which is of the form $\dfrac{1}{M}p^{\top}q$, where $p,q\in\mathbb{R}^{M}$. Thus, we have by Cauchy-Schwartz inequality,

[TABLE]

It can be shown that $p^{(m)}=\alpha^{3}\overline{\overline{\sigma}}_{i}^{(m)}(1-\overline{\overline{\sigma}}_{i}^{(m)})(1-2\overline{\overline{\sigma}}_{i}^{(m)})$, whereby $||p||_{\infty}\leq\alpha^{3}$.

Finally, we see that $q^{(m)}=t_{j}\left|\left|\Phi_{-i}^{(m)^{\top}}(\hat{w}_{i}-w_{i}^{*})\right|\right|_{2}^{2}\leavevmode\nobreak\ \geq\leavevmode\nobreak\ 0$ since $t_{j}\in[0,1]$. Therefore $||q||_{1}\leavevmode\nobreak\ =\leavevmode\nobreak\ q^{\top}\boldsymbol{1}$, where $\boldsymbol{1}\in\mathbb{R}^{M}$ is a vector of all ones. Moreover, since $\hat{w}_{i,S^{c}}=w_{i,S^{c}}^{*}=\boldsymbol{0}$, we note that

[TABLE]

Since $\gamma\in(0,1]$, so

[TABLE]

Therefore, we can invoke Lemma R2 when $||G_{i}^{M}||_{\infty}\leavevmode\nobreak\ \leq\leavevmode\nobreak\ \dfrac{\lambda_{M}}{4}$. Specifically, we then have for each $j$,

[TABLE]

This immediately yields $\dfrac{||R_{i}^{M}||_{\infty}}{\lambda_{M}}\leavevmode\nobreak\ \leq\leavevmode\nobreak\ \dfrac{25C_{\max}}{\alpha C_{\min}^{2}}\lambda_{M}d\leavevmode\nobreak\ .$ ∎

Theorem R4**.**

Let $M>\dfrac{80^{2}C_{\max}^{2}}{C_{\min}^{4}}\left(\dfrac{2-\gamma}{\gamma}\right)^{4}d^{2}\log(n)$, and $\lambda_{M}\geq\dfrac{8\alpha(2-\gamma)}{\gamma}\sqrt{\dfrac{\log(n)}{M}}$. Suppose the sample satisfies assumptions (41), (42), and (43). Consider any player $i\in[n]$. The following results hold with probability at least $1-2\exp(-C_{\alpha,\lambda}\lambda_{M}^{2}M)\to 1$ for $i$.

The corresponding $\ell_{1}$-regularized optimization problem has a unique solution, i.e., a unique set of neighbors for $i$. 2. 2.

The set of predicted neighbors of $i$ is a subset of the true neighbors. Additionally, the predicted set contains all true neighbors $j$ for which $|w_{ij}^{*}|\geq\dfrac{10}{\alpha^{2}C_{\min}}\sqrt{d}\lambda_{M}$. In particular, the set of true neighbors of $i$ is exactly recovered if

[TABLE]

Taking a union bound over players, our results imply that we recover the true signed neighborhoods for all players in the LAG with probability at least $1-2n\exp(-C_{\alpha,\lambda}\lambda_{M}^{2}M)$ .

Proof.

Since $\lambda_{M}\geq\dfrac{8\alpha(2-\gamma)}{\gamma}\sqrt{\dfrac{\log(n)}{M}}$, Lemma R1 holds. Thus, with high probability (as stated in the theorem statement), we obtain

[TABLE]

since $\gamma\in(0,1]$. Moreover, for the specified lower bound on sample size $M$, a simple computation shows

[TABLE]

Thus the conditions required for both Lemma R2 and Lemma R3 are satisfied. By our primal-dual construction, $\hat{w}_{i,S^{c}}=\boldsymbol{0}$. Furthermore, using (41), $\Lambda_{\min}(H_{i,SS}^{*M})>0$, and so $H_{i,SS}^{*M}$ is invertible. Separating the rows in the support of $i$ and others, we write (51) as

[TABLE]

These two equations can be combined into one as

[TABLE]

Recalling that $||\hat{\kappa}_{i,S}||_{\infty}<1$, we immediately get that $\lambda_{M}||\hat{\kappa}_{i,S^{c}}||_{\infty}$

[TABLE]

Since $\gamma\in(0,1]$ and $\lambda_{M}>0$, we immediately get $||\hat{\kappa}_{i,S^{c}}||_{\infty}<1$. Therefore, strict dual feasibility is established and (46) is verified. Then, using Lemma 1 of [45], we note that any optimal solution $\tilde{w}_{i}$ of (44) must have $\tilde{w}_{i,S^{c}}=\boldsymbol{0}$. In particular, we have $\hat{w}_{i,S^{c}}=\boldsymbol{0}$ as desired. Thus, we can focus on $\hat{w}_{i,S}$. We now prove uniqueness of $\hat{w}_{i}$ by showing that $\Lambda_{\min}\left(\hat{H}_{i,SS}^{M}\right)\leavevmode\nobreak\ >\leavevmode\nobreak\ 0$. Let $\Delta=\hat{w}_{i,S}-w_{i,S}^{*}\in\mathbb{R}^{d}$. Then, using Lemma R2, we have

[TABLE]

Note that

[TABLE]

Performing a Taylor expansion around $w^{*}_{i,S}$, and making arguments similar to the proof segment between (56) and (57) in Lemma 2, we can show that

[TABLE]

which is greater than 0. Therefore, $\hat{H}_{i,SS}^{M}$ is positive definite, and Lemma 1 of [45] guarantees that $\hat{w}_{i}$ is the unique optimal primal solution for (44).

We finally argue about the only remaining condition (47). In order for neighbor $j$ to be correctly recovered with sign, i.e., $\text{sign}(\hat{w}_{ij})=\text{sign}(w_{ij}^{*})$, it suffices to have

[TABLE]

Moreover to recover the neighborhood of $i$ exactly, it is sufficient to show

[TABLE]

which implies (60). We note that

[TABLE]

Using (61), it immediately follows that the neighborhood of $i$ is recovered with correct sign if

[TABLE]

∎