Learning to Manipulate a Commitment Optimizer

TL;DR

This paper investigates how a follower in a Stackelberg game can learn to manipulate a leader's commitments through interaction and querying, even without initial knowledge of the leader's payoffs, highlighting the potential risks for commitment optimizers.

Contribution

It demonstrates that followers can learn to manipulate a leader's commitments efficiently without prior payoff information, reducing the importance of the assumed information advantage.

Findings

Follower can learn manipulation strategies in polynomial time

Manipulation is possible without initial payoff knowledge

Polynomially many queries suffice for effective manipulation

Abstract

It is shown in recent studies that in a Stackelberg game the follower can manipulate the leader by deviating from their true best-response behavior. Such manipulations are computationally tractable and can be highly beneficial for the follower. Meanwhile, they may result in significant payoff losses for the leader, sometimes completely defeating their first-mover advantage. A warning to commitment optimizers, the risk these findings indicate appears to be alleviated to some extent by a strict information advantage the manipulations rely on. That is, the follower knows the full information about both players' payoffs whereas the leader only knows their own payoffs. In this paper, we study the manipulation problem with this information advantage relaxed. We consider the scenario where the follower is not given any information about the leader's payoffs to begin with but has to learn to manipulate by interacting with the leader. The follower can gather necessary information by querying the leader's optimal commitments against contrived best-response behaviors. Our results indicate that the information advantage is not entirely indispensable to the follower's manipulations: the follower can learn the optimal way to manipulate in polynomial time with polynomially many queries of the leader's optimal commitment.

Equations

$$\mathrm{BR}(\mathbf{x}):=\underset{j\in [n]}{\mathrm{argmax}}{u}^F(\mathbf{x},j)$$

$${\mathrm{BR}}^{-1}(j):=\left\{\mathbf{x}\in {\Delta }_m:j\in \mathrm{BR}(\mathbf{x})\right\}$$

$$(\mathbf{x},j)\in \underset{{\mathbf{x}}^{\prime }\in {\Delta }_m,j^{\prime }\in \mathrm{BR}({\mathbf{x}}^{\prime })}{\mathrm{argmax}}{u}^L({\mathbf{x}}^{\prime },j^{\prime }).$$

$$(\mathbf{x},j)\in \underset{{\mathbf{x}}^{\prime }\in {\mathrm{BR}}^{-1}(j^{\prime }),j^{\prime }\in [n]}{\mathrm{argmax}}{u}^L({\mathbf{x}}^{\prime },j^{\prime }).$$

$$\underset{{\tilde{u}}^F,\mathbf{x},j}{\max }$$

$$(\mathbf{x},j)\text{ is an SSE of }\tilde{\mathcal{G}}=(u^L,{\tilde{u}}^F).$$

$$u^L(\mathbf{x},j)\ge M_{[n]}:=\underset{\mathbf{y}\in {\Delta }_m}{\max }\underset{k\in [n]}{\min }{u}^L(\mathbf{y},k).$$

$$M_S:=\underset{\mathbf{x}\in {\Delta }_m}{\max }\underset{j\in S}{\min }{u}^L(\mathbf{x},j)\text{ and}{\mathcal{M}}_S:=\underset{\mathbf{x}\in {\Delta }_m}{\mathrm{argmax}}\underset{j\in S}{\min }{u}^L(\mathbf{x},j),$$

$$u^L(i,j)=\underset{\mathbf{x}\in {\widetilde{\mathrm{BR}}}^{-1}(j)}{\max }{u}^L(\mathbf{x},j)=\underset{\mathbf{x}\in {\Delta }_m}{\max }{u}^L(\mathbf{x},j)\ge \underset{i^{\prime }\in [m]}{\max }{u}^L(i^{\prime },j).$$

$${\widetilde{\mathrm{BR}}}^{-1}(k)=\{i\},{\widetilde{\mathrm{BR}}}^{-1}(j)={\Delta }_m,\text{and }{\widetilde{\mathrm{BR}}}^{-1}(j^{\prime })=\varnothing \text{ for all }{j}^{\prime }\in [n]\setminus \{j,k\}.$$

$$\underset{\mathbf{x}\in {\Delta }_m}{\max }$$

$$u^L(\mathbf{x},j)\ge M_{[n]}\text{em(i.e., (x,j) is inducible)}$$

$$u^L(\mathbf{x},j)\equiv {\gamma }_j\cdot {\mathbf{a}}_j\cdot \mathbf{x}+{\beta }_j\text{ for some }{\gamma }_j\in {\mathbb{R}}_{>0}\text{ and }{\beta }_j\in \mathbb{R},$$

$$u^L({\mathbf{x}}^{\ast },j)=M_J=M_{[n]},\text{ for all }j\in J.$$

$${\tilde{u}}^L(\mathbf{x},j)=\{\begin{array}{ll}{\mathbf{a}}_j\cdot \mathbf{x}-{\mathbf{a}}_j\cdot {\mathbf{x}}^{\ast },& \text{ if }j\in J\setminus \{k\};\\ {\mathbf{a}}_k\cdot \mathbf{x}-{\mathbf{a}}_k\cdot \mathbf{y},& \text{ if }j=k;\\ 1,& \text{ otherwise. }\end{array}$$

$$u^L(\mathbf{x},n)\equiv \sum _{i=1}^m{u}^L(i,n)\cdot x_i\equiv \sum _{i=1}^{m_1-1}\left(u^L(i,n)-u^L(m,n)\right)\cdot x_i+u^L(m,n).$$

$$\left(u^L(1,n)-u^L(m,n)\right)\cdot x_1^1=\cdots =\left(u^L(m_1-1,n)-u^L(m,n)\right)\cdot x_{m_1-1}^{m_1-1}.$$

$$\underset{\mathbf{x}\in {\mathcal{M}}_S}{\max }\underset{j\in \widetilde{\mathrm{BR}}(\mathbf{x})}{\max }{u}^L(\mathbf{x},j)<M_S,$$

$${\Gamma }_i:=\left\{\mathbf{x}\in {\Delta }_m:1-x_i=\sum _{k=m_1}^m{x}_k\right\},$$

$${\tilde{u}}_{\mathbf{g}}^F(\mathbf{x},j):=\{\begin{array}{ll}{\sum }_{i=m_1}^m{x}_i\cdot \mu (i,j)& \text{ if }j\in [n-1];\\ {\sum }_{i=1}^{m_1-1}{x}_i\cdot g_i+W\cdot {\sum }_{i=m_1}^m{x}_i& \text{ if }j=n.\end{array}$$

$${\mathbf{x}}^i\in \underset{\mathbf{x}\in {\Gamma }_i\cap f_{\mathbf{g}}^{-1}(n)}{\mathrm{argmin}}{x}_i,$$

$$I_{\mathbf{g}}:=\left\{i\in [m_1-1]:({\mathbf{x}}^i,n)\text{ is an SSE in game }\left(u^L,{\tilde{u}}_{\mathbf{g}}^F\right)\right\}.$$

$$u^L(\mathbf{x},\ell )\le M_{\{\ell \}}<M_{\{n\}}$$

$$\mu (\mathbf{x},j):=\{\begin{array}{ll}0,& \text{if }j\in S;\\ {\mathbf{b}}_j\cdot \mathbf{x}-c_j,& \text{otherwise}.\end{array}$$

$$u^L(\mathbf{x},j)\ge M_S⟺{\mathbf{b}}_j\cdot \mathbf{x}\le c_j.$$

$${\tilde{u}}^L(\mathbf{x},j)=\{\begin{array}{ll}\frac{{\gamma }_j}{{\gamma }_1}\cdot {\mathbf{a}}_j\cdot \mathbf{x}+\frac{{\beta }_j-{\beta }_1}{{\gamma }_1},& \text{ if }j\in \hat{J};\\ W,& \text{ otherwise},\end{array}$$

$$\hat{J}:=\left\{j\in [n]:\underset{\mathbf{x}\in {\Delta }_m}{\min }{u}^L(\mathbf{x},j)<M_{\{1\}}\right\}.$$

$${\tilde{M}}_S:=\underset{\mathbf{x}\in {\Delta }_m}{\max }\underset{j\in S}{\min }{\tilde{u}}^L(\mathbf{x},j)$$

$$\{\begin{array}{l}{\gamma }_1\cdot {\mathbf{a}}_1\cdot {\mathbf{x}}_1+{\beta }_1={\gamma }_j\cdot {\mathbf{a}}_j\cdot {\mathbf{y}}_1+{\beta }_j\\ {\gamma }_1\cdot {\mathbf{a}}_1\cdot {\mathbf{x}}_2+{\beta }_1={\gamma }_j\cdot {\mathbf{a}}_j\cdot {\mathbf{y}}_2+{\beta }_j\end{array}$$

$$u^L(\mathbf{x},1)=u^L(\mathbf{y},2)=M_{\{1,2\}}.$$

$${\tilde{u}}_d^F(\mathbf{x},j)=\{\begin{array}{ll}-1,& \text{if }j\in [n]\setminus \{1,2\};\\ 0,& \text{if }j=1;\\ d-{\mathbf{a}}_2\cdot \mathbf{x},& \text{if }j=2.\end{array}$$

Full text

††footnotetext: ∗Corresponding author.

1 Introduction

Strategy commitment is a useful tactic in many game-theoretic scenarios. In anticipation that the other player, i.e., the follower, will respond optimally, a commitment optimizer, i.e., the leader, picks a strategy that maximizes their own payoff. The interaction between the leader and the follower is often modeled and known as a Stackelberg game [von Stackelberg, 1934; Von Stengel and Zamir, 2010]. The equilibrium of the game, called the Stackelberg equilibrium, captures the leader’s optimal commitment. It is well-known that the leader’s first-mover position comes with a payoff benefit: an optimal commitment always yields a higher payoff (often strictly higher) for the leader than what they obtain in any Nash equilibrium of the same game [Von Stengel and Zamir, 2010].

Yet, this first-mover advantage is not without a caveat. The computation of an optimal commitment relies crucially on the follower’s payoff information. This strong reliance offers the follower a means to manipulate the leader’s commitment. A series of recent studies considered this issue and investigated how a follower can induce an equilibrium different from the original one by misreporting their payoffs [Gan et al., 2019b; a; Nguyen and Xu, 2019; Birmpas et al., 2021; Chen et al., 2022b]. It is shown that finding an optimal manipulation is computationally tractable and there is a large space of outcomes that are realizable through manipulations. In the worst case, a manipulation may completely defeat the first-mover advantage of the leader and cause a significant payoff loss.111Birmpas et al. [2021] showed that any outcome can be induced as a Stackleberg equilibrium as long as it offers the leader at least the maximin value of the game—a lower bound of the leader’s Nash payoff, in contrast to the upper bound promised by an optimal commitment when the follower behaves truthfully. Moreover, even if the leader is well aware of the possibility of such manipulations, they face an NP-hard problem to compute an optimal mechanism to counteract [Gan et al., 2019b].

A warning to commitment optimizers, the risk these findings indicate may appear to be alleviated to some extent by a strict information advantage required by the manipulations. That is, the follower knows the full information about both players’ payoffs, whereas the leader only knows their own. This may not be the case in practice. In this paper, we consider a setting with this information asymmetry relaxed, where neither the leader nor the follower knows any payoff information of the opponent to begin with. The follower has to learn to manipulate by interacting with the leader and can gather necessary information by querying the leader’s optimal commitments against contrived payoff functions. We are interested in understanding whether the follower can efficiently learn to solve the optimal manipulation problem.

More specifically, we assume that the follower has query access to the leader’s optimal commitment: there is an equilibrium oracle which answers whether a certain Stackelberg equilibrium can be induced by a given (fake) payoff function of the follower. The query access resembles an information exchange process during the course of interaction: the follower (mis)reports a payoff function to the leader, and the leader reacts by committing optimally with respect to the report; the optimal commitment is then observed by the follower. Payoff reporting may take the form of direct information exchange. For example, in online platforms, users (follower) set up their profiles and the platform (leader) offers personalized recommendations or pricing based on the information provided.222https://medium.com/swlh/why-is-your-friend-getting-a-cheaper-uber-fare-than-you-ai-and-the-new-frontier-in-dynamic-pricing-2b7d908deed0 Additionally, it can also be realized via another layer of active learning from the leader’s side, e.g., in green security games, the defender (leader) learns the optimal patrolling strategy by interacting with poachers (followers) [Fang and Nguyen, 2016].

Our main result, presented as follows, is an affirmative answer to the question asked above.

Main Theorem** (informal).**

With access to an equilibrium oracle, a follower can learn an optimal payoff function to misreport in polynomial time and via polynomially many queries. The payoff function induces a Stackelberg equilibrium that maximizes the follower’s (real) payoff among all inducible equilbria (i.e., equilibria that can be induced by some fake payoff matrix of the follower).

The result indicates that the information advantage is not entirely indispensable to the follower’s manipulation, so it may not be safe to take it as a protection against manipulations. Indeed, the issue is fairly widespread. The flourish of e-commerce and other online platforms, including various financial activities on crypto-currencies [Chen et al., 2022a], offers many testbeds and realistic application areas for the problem we study. In these domains, the leader interacts individually with a large number of different followers and aims to commit optimally against each of them. A follower who intends to manipulate can easily forge pseudonym identities in a short period of time at low cost and without being detected by the leader (e.g., by creating multiple accounts for a web-based service such as crypto-currency). Adding to the fact, the manipulations we consider are imitative—a term coined by Gan et al. [2019b] meaning that the follower keeps behaving in accordance with the reported payoff function. Such manipulations are almost impossible for the leader to detect in many cases. Due caution is needed when one seeks to exploit the power of commitment in these domains.

Our approach to deriving the main result consists of two main components: (1) learning the gradient information of the leader’s payoff function, and (2) constructing strategically equivalent games with this information to compute the follower’s best manipulation strategy. The latter requires computing a payoff matrix of the follower to induce her maximin value in the original game. Even in the full information setting, this problem requires a non-trivial approach in order to derive an efficient solution Birmpas et al. [2021]. In our partial information setting, additional difficulties come from the fact that we cannot fully recover all gradient information of the leader’s payoff function. We note that the equilibrium oracle does not directly reveal the leader’s payoff information. Therefore, to acquire useful information from the oracle, it requires carefully designed queries that expose strategy profiles of interest as Stackelberg equilibria. This task becomes more challenging, as the numbers of players’ actions increase.

1.1 Related Work

Our work directly relates to the recent line of work on follower deception in Stackelberg games as we mentioned above. This line of work is motivated by an active learning approach to finding an optimal strategy to commit to [Letchford et al., 2009; Balcan et al., 2015; Blum et al., 2014; Roth et al., 2016; Peng et al., 2019], which asks whether the leader’s optimal commitment can be learned with query access to the follower’s best response. Gan et al. [2019b] first pointed out that this approach leads to an untruthful mechanism that can be manipulated by the following imitating responses as if they have a different payoff function. They also showed that designing an optimal mechanism to counteract the follower’s manipulation is in general NP-hard even to find an approximate solution. The hardness contrasts the tractability of the computation of an optimal commitment in the full information setting, as shown in an early work of Conitzer and Sandholm [2006]. Gan et al. [2019a] considered a security game scenario and showed that, under mild assumptions, in a Stackelberg security game the follower’s optimal manipulation is always to misreport payoffs that make the game zero-sum; the leader gains only their maximin payoff as a result. This result does not hold true in general bi-matrix Stackelberg games, but the later result of Birmpas et al. [2021] indeed also revealed a general connection between payoff manipulations in Stackelberg games and the leader’s maximin payoff. As we will discuss in detail later, this connection is also one of the cornerstones of our main result. Nguyen and Xu [2019] studied a similar security game scenario and also considered the follower’s manipulation strategy when their payoff reporting is restricted in a ball of the true payoff. More recently, Chen et al. [2022b] further extended the line of work to extensive-form games. In another line of work Kolumbus and Nisan [2022b; a] studied the Nash equilibrium of the meta-game when multiple players attempt to manipulate simultaneously.

Besides the above line of work, in a more recent study, Haghtalab et al. [2022] approached manipulations in Stackelberg games via a repeated game model and interpreted the follower’s manipulations as a strategic behavior resulting from their far-sightedness. A framework is proposed in this study to derive efficient learning algorithm for a more patient leader (who does not discount future rewards) to induce truthful best responses from the less patient non-myopic follower (who discounts future rewards). Our setting is analogous to the opposite, where a more patient follower plays with a less patient leader. The patient follower explores the leader’s payoff structure until having learned the optimal manipulation and behaves accordingly afterwards to exploit the leader. We also note that similar interactions against non-myopic bidders have also been extensively explored in the online auction literature Amin et al. [2013; 2014]; Mohri and Medina [2014]; Liu et al. [2018]; Abernethy et al. [2019]; Golrezaei et al. [2021].

2 Preliminaries

Stackelberg games are a standard framework for studying strategy commitment in game theory. In a Stackelberg game, a leader commits to a strategy and a follower best responds to this commitment. We consider general bi-matrix games in this paper. A bi-matrix game $\mathcal{G}=(u^{L},u^{F})$ is given by two matrices $u^{L},u^{F}\in\mathbb{R}^{m\times n}$, which specify the leader’s and the follower’s payoffs, respectively. The leader has $m$ actions (i.e., pure strategies) at their disposal, each corresponding to a row of the payoff matrices; and the follower has $n$ actions, each corresponding to a column. The entries $u^{L}(i,j)$ and $u^{F}(i,j)$ are the payoffs of the leader and the follower when a pair $(i,j)\in[m]\times[n]$ of actions is played.333For any positive integer $x$ we write $[x]\coloneqq\{1,\dots,x\}$.

In the mixed strategy setting, the leader can further randomize their actions and the resulting distribution over the actions is called a mixed strategy, e.g., $\mathbf{x}\in\Delta_{m}\coloneqq\left\{\mathbf{x}\in\mathbb{R}_{\geq 0}^{m}:\sum_{i\in[m]}x_{i}=1\right\}$. Slightly abusing notation, we denote by $u^{L}(\mathbf{x},j)=\sum_{i\in[m]}x_{i}\cdot u^{L}(i,j)$ the expected payoff of the leader for a strategy profile $(\mathbf{x},j)$, and similarly denote by $u^{F}(\mathbf{x},j)=\sum_{i\in[m]}x_{i}\cdot u^{F}(i,j)$ the expected payoff of the follower. We will refer to a payoff matrix and the corresponding payoff function interchangeably throughout this paper.

A (pure) best response of the follower is then given by $j\in\operatorname{BR}(\mathbf{x})$, where

[TABLE]

is called the follower’s best response set, or a BR-correspondence as a function $\operatorname{BR}:\Delta_{m}\to 2^{[n]}$. In most cases, it is without loss of generality to consider only pure strategy responses, because there always exists an optimal strategy that is pure. Hence, unless otherwise specified, all best responses are pure strategies throughout. It will also be useful to define the inverse function of $\operatorname{BR}$: for any $j\in[n]$,

[TABLE]

is the set of leader strategies that incentivize the follower to best-respond $j$.

The optimal commitments of the leader are captured by the following optimization, with the assumption that the follower breaks ties by picking a $j\in\operatorname{BR}(\mathbf{x})$ in favor of the leader when there are multiple best responses in $\operatorname{BR}(\mathbf{x})$:

[TABLE]

The strategy profile $(\mathbf{x},j)$ is called a strong Stackelberg equilibrium (SSE). Alternatively, using the inverse function of $\operatorname{BR}$ gives the following equivalent definition of an SSE:

[TABLE]

SSE is the most widely used solution concept in the literature on Stackelberg games. The optimistic tie-breaking assumption adopted by it is justified by noting that this tie-breaking behavior can often be induced by an infinitesimal perturbation in the leader’s strategy [von Stengel and Zamir, 2004].

Definition 1** (SSE and SSE response).**

A strategy profile $(\mathbf{x},j)\in\Delta_{m}\times[n]$ is said to be an SSE of a game $\mathcal{G}=(u^{L},u^{F})$ if and only if Eq. 1 (or equivalently, Eq. 2) holds. An action $j$ of the follower is called an SSE response of $\mathcal{G}$ if and only if $(\mathbf{x},j)$ is an SSE of $\mathcal{G}$ for some $\mathbf{x}\in\Delta_{m}$.

2.1 Equilibrium Manipulation via Payoff Misreporting

According to the above definition, the optimal commitment of the leader, or the SSE, is a function of the follower’s payoff matrix. When this payoff information is private to the follower, the follower has a chance to manipulate the leader’s commitment by reporting a fake payoff matrix. At a high-level, to find out the optimal way to manipulate amounts to solving the following optimization problem:

[TABLE]

Birmpas et al. [2021] showed that this problem is tractable in the full information setting (i.e., when $u^{L}$ is known to the follower) and presented an elegant characterization of strategy profiles that satisfy Eq. 3a. This characterization, summarized in Theorem 2.1 below, forms a foundation of our technical results. We follow the terminology by Birmpas et al. [2021] and define the inducibility as follows. Note that Eq. 3a also means that the manipulation is imitative: the follower is required to respond according to the reported payoff $\tilde{u}^{F}$, and this “best response” decides their equilibrium utility.

Definition 2** (Inducibility).**

A payoff matrix $\tilde{u}^{F}\in\mathbb{R}^{m\times n}$ induces a strategy profile $(\mathbf{x},j)$ (with respect to $u^{L}$) if and only if $(\mathbf{x},j)$ is an SSE of $(u^{L},\tilde{u}^{F})$. Moreover, $(\mathbf{x},j)$ is said to be inducible (with respect to $u^{L}$) if and only if it is induced by some $\tilde{u}^{F}$.

Theorem 2.1** (Birmpas et al. 2021).**

A strategy profile $(\mathbf{x},j)$ is inducible if and only if

[TABLE]

Moreover, a payoff matrix $\tilde{u}^{F}$ that induces $(\mathbf{x},j)$ can be computed in polynomial time.

The above result establishes an interesting connection between the follower’s optimal manipulation and the leader’s maximin value $M_{[n]}$. Intuitively, to induce $(\mathbf{x},j)$, the follower can respond in a way that is completely adversarial against the leader (whereby the leader only gets $M_{[n]}$) unless the leader plays $\mathbf{x}$. Hereafter, we extend the notation $M_{[n]}$ to every subset $S\subseteq[n]$ of the follower’s actions, and define

[TABLE]

which are the leader’s maximin payoff and the set of maximin strategies when the follower’s responses are restricted in $S$. These two notations will be frequently used throughout the paper.

2.2 Main Problem: Learning to Manipulate with SSE Oracle

We consider the learning version of the follower’s optimal manipulation problem, where the leader’s payoff matrix is unknown to the follower (i.e., unknown to us). Instead, the follower only has query access to an SSE oracle, denoted $\mathcal{A}_{\textnormal{SSE}}$, which is able to answer whether a given solution $(\tilde{u}^{F},\mathbf{x},j)$ to the above optimization problem satisfies Eq. 3a or not—in other words, whether $(\mathbf{x},j)$ is induced by $\tilde{u}^{F}$.444The reason that we also specify $(\mathbf{x},j)$ as part of the input to $\mathcal{A}_{\textnormal{SSE}}$ is to sidestep the tricky case where there are multiple SSEs. Our results do not apply to the setting where we do not have the power to specify a particular SSE. We leave this setting as an interesting open problem. See our discussion in Section 6. Conceptually, when the follower can interact repeatedly with the leader, they can try reporting different payoff matrices $\tilde{u}^{F}$ and observe the leader’s optimal commitment against these matrices. The SSE oracle abstract this process. Note that in our model the leader is unaware of the fact that the follower keeps changing their payoffs throughout the process. Rather, the leader thinks that they are interacting with different followers and, as we discussed earlier, this applies to scenarios where the follower can easily forge a large number of fake identities to elicit the leader’s payoff information. To put it differently, the leader commits to playing the optimal commitment against every reported payoff matrix.

Definition 3** (SSE oracle).**

Given a matrix $\tilde{u}^{F}$ and a strategy profile $(\mathbf{x},j)\in\Delta_{m}\times[n]$, the SSE oracle, denoted $\mathcal{A}_{\textnormal{SSE}}$, outputs whether $\tilde{u}^{F}$ induces $(\mathbf{x},j)$ or not.

We consider the time and query complexity with respect to the bit-size of the leader’s payoff matrix and assume that its entries are rational numbers. Throughout, when we say that something can be computed efficiently, we mean that it can be computed with polynomially many queries to $\mathcal{A}_{\textnormal{SSE}}$ and in polynomial time. Our algorithms will make frequent use of binary search to find the exact value of an unknown rational number. We note that this can be done exactly in time polynomial in the bit-size of the target number, using a standard approach that searches on the Stern-brocot tree [Graham et al., 1989].

3 Warm-up and Approach Overview

In this section, we provide an overview of our approach to solving the problem defined above. We start with a warm-up, with two examples showing how the SSE oracle can be utilized to obtain some basic information that will be useful throughout the paper.

3.1 Warm-up with $\mathcal{A}_{\textnormal{SSE}}$

The oracle $\mathcal{A}_{\textnormal{SSE}}$ does not directly reveal any payoff values, but it exposes payoff information about SSEs. For example, if through the oracle we can confirm that two strategy profiles $(\mathbf{x},j)$ and $(\mathbf{x}^{\prime},j^{\prime})$ are both SSEs, then we know that $u^{L}(\mathbf{x},j)=u^{L}(\mathbf{x}^{\prime},j^{\prime})$. Hence, a main approach to obtaining information via $\mathcal{A}_{\textnormal{SSE}}$ is by designing payoff matrices that induce strategy profiles of interest as SSEs. In particular, in order for a profile $(\mathbf{x},j)$ to be an SSE, a necessary condition according to (2) is that $\mathbf{x}\in\operatorname*{argmax}_{\mathbf{x}^{\prime}\in\widetilde{\operatorname{BR}}^{-1}(j)}u^{L}(\mathbf{x}^{\prime},j)$ with respect to some follower action $j$ and the BR-correspondence $\widetilde{\operatorname{BR}}^{-1}(j)$ of a fake payoff matrix. Using this necessary condition and $\mathcal{A}_{\textnormal{SSE}}$, we can easily, and efficiently, identify the following handy information; we henceforth assume that they are known in the remainder of the paper.

Observation 3.1**.**

With query access to $\mathcal{A}_{\textnormal{SSE}}$, for every $j\in[n]$, the set $I_{j}\coloneqq\operatorname*{argmax}_{i\in[m]}u^{L}(i,j)$ of the leader’s (pure) best responses against $j$ can be computed efficiently. Hence, $\mathcal{M}_{\{j\}}$, which is the convex hull of $I_{j}$, can also be computed efficiently.

Specifically, to decide whether $i\in I_{j}$ for a pure strategy $i\in[m]$, we can construct a matrix $\tilde{u}^{F}$ such that $\tilde{u}^{F}(i,j)=0$ and $\tilde{u}^{F}(i,j^{\prime})=-1$ for all $j^{\prime}\neq j$. This way $j$ is the strictly dominant strategy of the follower, so we have $\widetilde{\operatorname{BR}}^{-1}(j)=\Delta_{m}$ and $\widetilde{\operatorname{BR}}^{-1}(j^{\prime})=\emptyset$ for all $j^{\prime}\neq j$. We then query $\mathcal{A}_{\textnormal{SSE}}$ to check whether $\tilde{u}^{F}$ induces $(i,j)$ as an SSE. A “yes” answer implies that

[TABLE]

So $i\in I_{j}$. A “no” answer implies the existence of $\mathbf{x}\in\widetilde{\operatorname{BR}}^{-1}(j)=\Delta_{m}$ such that $u^{L}(i,j)<u^{L}(\mathbf{x},j)$. It follows that $u^{L}(i,j)<u^{L}(\mathbf{x},j)\leq\max_{i^{\prime}\in[m]}u^{L}(i^{\prime},j)$. Hence, $i\notin I_{j}$.

Observation 3.2**.**

For every $i\in[m]$ and $j,k\in[n]$, the relation (i.e., $>$, $=$, or $<$) between $M_{\{j\}}$ and $u^{L}(i,k)$ can be decided efficiently with query access to $\mathcal{A}_{\textnormal{SSE}}$.

To decide the above relation, we slightly modify $\tilde{u}^{F}$ defined for deriving 3.1; we let $\tilde{u}^{F}(i,k)=0$ and keep all other entries the same. This gives the following BR-correspondence of $\tilde{u}^{F}$:

[TABLE]

Consequently, the best strategies to induce responses $j$ and $k$ yield payoffs $M_{\{j\}}$ and $u^{L}(i,k)$, respectively, for the leader. We can then query $\mathcal{A}_{\textnormal{SSE}}$ to check if $(i^{\prime},j)$ and $(i,k)$ are SSEs for some $i^{\prime}\in I_{j}$ to decide the relation between these two payoffs.

3.2 Approach Overview

Now we present an overview of our approach. Given the characterization by Theorem 2.1, the problem we want to solve, formulated as Problem 3, boils down to solving the following linear program (LP) for every $j\in[n]$ and selecting the one with the maximum optimal value.

[TABLE]

With only access to $\mathcal{A}_{\textnormal{SSE}}$, we cannot hope to learn the exact payoff function $u^{L}$ to solve the above LP. Hence, the idea is to learn a strategically equivalent representation of $u^{L}$, that is, a vector $\mathbf{a}_{j}=(a_{j1},\dots,a_{j{m}})\in\mathbb{Q}^{m}$ such that

[TABLE]

where $\mathbf{a}_{j}\cdot\mathbf{x}$ denotes the inner product of $\mathbf{a}_{j}$ and $\mathbf{x}$. Intuitively, $\mathbf{a}_{j}$ indicates the direction of the gradient of $u^{L}(\cdot,j)$. As we demonstrate in Lemma 3.3, knowing $\mathbf{a}_{j}$ alone (without $\gamma_{j}$ and $\beta_{j}$) allows us to reduce the LP to an inducibility problem: decide whether a given strategy profile $(\mathbf{x},j)$ is inducible or not. To solve the inducibility problem means comparing $u^{L}(\mathbf{x},j)$ with the maximin payoff $M_{[n]}$ and this requires constructing a $\tilde{u}^{F}$ that induces either $(\mathbf{x},j)$ or a strategy profile that gives the maximin payoff.

In more detail, these procedures are summarized in Figure 1, which also include a special treatment of a degenerate case (i.e., when $M_{\{j\}}=M_{[n]}$) that prevents us from even learning $\mathbf{a}_{j}$. The detailed implementations of Steps 1 and 2 are presented in the next sections: learning $\mathbf{a}_{j}$ in Section 4, and learning $J$ and $\mathbf{x}^{*}$ in Section 5. The reason that LP (5) can be solved efficiently in Step 3 is given by Lemma 3.3.

Lemma 3.3**.**

Suppose that there is an efficient algorithm that decides correctly whether any given strategy profile $(\mathbf{y},k)\in\Delta_{m}\times[n]$ is inducible or not. Then given $\mathbf{a}_{j}$ satisfying (6), LP (5) can be solved in polynomial time with query access to $\mathcal{A}_{\textnormal{SSE}}$. The same can be achieved without $\mathbf{a}_{j}$ if $M_{\{j\}}=M_{[n]}$.

Proof.

In the case where $M_{\{j\}}=M_{[n]}$, we have $u^{L}(\mathbf{x},j)\geq M_{[n]}$ if and only if $\mathbf{x}\in\mathcal{M}_{\{j\}}$. By 3.1, LP (5) is then equivalent to solving $\max_{\mathbf{x}\in\Delta_{m}}u^{F}(\mathbf{x},j)$, subject to $x_{i}=0$ for all $i\notin I_{j}$, which is an LP with all parameters known. Hence, LP (5) can be solved efficiently.

In the case where $M_{\{j\}}\neq M_{[n]}$, we are given $\mathbf{a}_{j}$. We can rewrite the constraint Eq. 5a as $\mathbf{a}_{j}\cdot\mathbf{x}\geq d_{j}^{*}$, where $d_{j}^{*}=(M_{[n]}-\beta_{j})/\gamma_{j}$. Note that $d_{j}^{*}$ is still unknown since $\beta_{j}$ and $\gamma_{j}$ are unknown. However, now that there is an algorithm to solve the inducibility problem, we can learn $d_{j}^{*}$ using binary search as follows. For any given $d$, pick arbitrary $\mathbf{x}\in\Delta_{m}$ such that $\mathbf{a}_{j}\cdot\mathbf{x}=d$. If $(\mathbf{x},j)$ is inducible then we know that $u^{L}(\mathbf{x},j)\geq M_{[n]}$ by Theorem 2.1 and hence, $d\geq d_{j}^{*}$; otherwise, we know that $d<d_{j}^{*}$. Knowing $d_{j}^{*}$, we can then solve LP (5) efficiently. ∎

{restatable}

theoremthmequgame

Suppose that the following elements are given: a vector $\mathbf{a}_{j}$ satisfying (6) for every $j\in[n]$ such that $M_{\{j\}}\neq M_{[n]}$; moreover, $J\subseteq[n]$ and $\mathbf{x}^{*}\in\Delta_{m}$ satisfying (7). Then for every $j\in[n]$, LP (5) can be solved in polynomial time with query access to $\mathcal{A}_{\textnormal{SSE}}$.

Proof sketch.

By Lemma 3.3, LP (5) reduces to deciding the inducibility of any given strategy profile $(\mathbf{y},k)$. We show that this inducibility problem can be efficiently solved. Specifically, we argue that the following algorithm produces an inducibility witness $\tilde{u}^{F}$ for $(\mathbf{y},k)$ in polynomial time. That is, $(\mathbf{y},k)$ is inducible (with respect to $u^{L}$) if and only if it is an SSE in $(u^{L},\tilde{u}^{F})$. Hence, by querying $\mathcal{A}_{\textnormal{SSE}}$ to check whether $(\mathbf{y},k)$ is an SSE of $(u^{L},\tilde{u}^{F})$, we can decide the inducibility of $(\mathbf{y},k)$. The stated result then follows.

{boxedtext}

Construct an inducibility witness $\tilde{u}^{F}$ of $(\mathbf{y},k)$:

For every $j$ such that $M_{\{j\}}=M_{[n]}$, since $\mathbf{a}_{j}$ is not given, let $\mathbf{a}_{j}$ be a vector in $\mathbb{R}^{m}$ such that: $a_{ji}=1$ if $i\in\mathcal{M}_{\{j\}}$, and $a_{ji}=0$ otherwise.

• 2.

Construct a payoff matrix $\tilde{u}^{L}\in\mathbb{R}^{m\times n}$ corresponding to the following payoff function:

[TABLE]

• 3.

Decide if $(\mathbf{y},k)$ is inducible with respect to $\tilde{u}^{L}$:

• –

If it is inducible, output a payoff matrix $\tilde{u}^{F}\in\mathbb{R}^{m\times n}$ such that $(\mathbf{y},k)$ is an SSE in $(\tilde{u}^{L},\tilde{u}^{F})$;

• –

Otherwise, output an arbitrary $\tilde{u}^{F}\in\mathbb{R}^{m\times n}$.

Namely, the above algorithm constructs a “surrogate” matrix $\tilde{u}^{L}$ using the given information $\mathbf{a}_{j}$, $J$, and $\mathbf{x}^{*}$. Then a witness $\tilde{u}^{F}$ is produced using $\tilde{u}^{L}$. The polynomial run-time of the algorithm is readily seen. In particular, Step 3 can be done in polynomial time according to Theorem 2.1. (Note that all parameters of $\tilde{u}^{L}$ is known, so this is a full-information setting.) It can also be proven that $(\mathbf{y},k)$ is inducible if and only if it is an SSE in $(u^{L},\tilde{u}^{F})$, where $\tilde{u}^{F}$ is the matrix produced by the above algorithm. The details can be found in Appendix A. ∎

4 Learning $\mathbf{a}_{j}$ - Directions towards which Leader’s Utility Increases

For notational simplicity, we present an algorithm for learning $\mathbf{a}_{n}$ instead of $\mathbf{a}_{j}$. The cases with other $\mathbf{a}_{j}$’s are analogous and can be handled by appropriate relabeling.

To learn $\mathbf{a}_{n}$, the high-level idea is to construct a BR-correspondence $\widetilde{\operatorname{BR}}$, such that the boundary of $\widetilde{\operatorname{BR}}^{-1}(n)$ is aligned with a hyperplane with norm vector $\mathbf{a}_{n}$. Since we are searching “in the dark” and only have access to $\mathcal{A}_{\textnormal{SSE}}$, we scan through possible positions of the boundary in the hope of a position where all the points on the boundary are SSE strategies. Take Figure 2 as an example, we want to adjust the two red points defining the boundary to a position where both points are SSE strategies (which form SSEs along with response $n$ of the follower). The boundary then aligns with the contour of $u^{L}(\cdot,n)$ and its norm vector aligns with $\mathbf{a}_{n}$.

The above idea is formalized in Lemma 4.1 below, which also considers degenerate cases where $\mathbf{a}_{n}$ is parallel to a facet of $\Delta_{m}$ (e.g., when $\mathbf{a}_{n}$ is parallel to an edge of the simplex in Figure 2). Special treatment is needed for such degenerate cases, as it shall be clear in the sequel. For ease of description, we reorder the leader’s actions according to 4.1 throughout this section. (The payoff information needed for the reordering is known because of 3.1.) Specifically, in the case where $m_{1}=1$, we can simply let $\mathbf{a}_{n}=\mathbf{0}$ without further learning it, so we can assume that $m_{1}>1$.

Observation 4.1**.**

Without loss of generality, we can assume that $u^{L}(m_{1},n)=u^{L}(m_{1}+1,n)=\cdots=u^{L}(m,n)=M_{\{n\}}$, $1<m_{1}\leq m$, and $u^{L}(k,n)<M_{\{n\}}$ for all $k=1,\dots,m_{1}-1$.

{restatable}

lemmalmmrefpointsconditions

Suppose that the following properties hold for strategies $\mathbf{x}^{1},\dots,\mathbf{x}^{m_{1}-1}\in\Delta_{m}$:

• (a)

$u^{L}(\mathbf{x}^{1},n)=\dots=u^{L}(\mathbf{x}^{m_{1}-1},n)$;

• (b)

$x_{i}^{i}>0$ for all $i\in[m_{1}-1]$; and

• (c)

$1-x_{i}^{i}=\sum_{k=m_{1}}^{m}x_{k}^{i}$ for all $i\in[m_{1}-1]$.

Let $\mathbf{b}=\left(-\frac{1}{x_{1}^{1}},-\frac{1}{x_{2}^{2}},\cdots,-\frac{1}{x_{m_{1}-1}^{m_{1}-1}},0,\cdots,0\right)$. Then there exists $\gamma\in\mathbb{R}_{>0}$ and $\beta\in\mathbb{R}$ such that $u^{L}(\mathbf{x},n)=\gamma\cdot\mathbf{b}\cdot\mathbf{x}+\beta$ for all $\mathbf{x}\in\Delta_{m}$.

Proof.

Recall that by 4.1, $u^{L}(m_{1},n)=\cdots=u^{L}(m,n)$. Hence,

[TABLE]

Given Property (c), for each $i\in[m_{1}-1]$, we have $u^{L}(\mathbf{x}^{i},n)=\left(u^{L}(i,n)-u^{L}(m,n)\right)\cdot x_{i}^{i}+u^{L}(m,n)$, so Property (a) implies that:

[TABLE]

Let $\gamma=-\left(u^{L}(1,n)-u^{L}(m,n)\right)\cdot x_{1}^{1}$ and $\beta=u^{L}(m,n)$. Hence, $\gamma>0$, and since $x_{i}^{i}>0$ by Property (b), we get that $u^{L}(\mathbf{x},n)\equiv\gamma\cdot\mathbf{b}\cdot\mathbf{x}+\beta$. ∎

Following 4.1, we aim to find $m_{1}-1$ critical points $\mathbf{x}^{1},\dots,\mathbf{x}^{m_{1}-1}$ with the listed properties. In Figure 2, These are the red points defining the boundary hyperplane shared by area $C$. To ensure Property (a), the critical points need to form SSEs with action $n$. Ideally, we can design the BR-correspondence in a way such that the follower responds to strategies in $C$ (as in Figure 2) with a bad action to the leader, so that the best strategy of the leader in $C$ does not outperform the critical points. Sometimes this requires using more than one follower response to “cover” $C$. We introduce the following useful concept called maximin-cover, or cover for short.

Definition 4** (Cover).**

A payoff matrix $\mu\in\mathbb{R}^{m\times n}$ of the follower is said to be a maximin-cover (or cover) of $S\subseteq[n]$ if and only if

[TABLE]

where $\widetilde{\operatorname{BR}}$ denotes the BR-correspondence of $\mu$. It is said to be a proper cover of $S$ if it holds in addition that $S\cap\widetilde{\operatorname{BR}}(\mathbf{x})=\emptyset$ for all $\mathbf{x}\in\Delta_{m}$.

We arrange the BR-correspondence in region $C$ according to a cover of $\{n\}$, in an attempt to make the leader’s maximum attainable payoff in $\widetilde{\operatorname{BR}}^{-1}(n)$ surpass that in $C$. To see how this could work, consider moving the hyperplane $\mathbf{a}_{n}\cdot\mathbf{x}=d$ in Figure 2 towards the top. As it approaches the vertex at the top, $\widetilde{\operatorname{BR}}^{-1}(n)$ and $C$ will approach $\Delta_{m}$ and $\mathcal{M}_{\{n\}}$, respectively. The maximum attainable payoffs in $\widetilde{\operatorname{BR}}^{-1}(n)$ and $C$ then approaches $\max_{\mathbf{x}\in\Delta_{m}}u^{L}(\mathbf{x},n)=M_{\{n\}}$ and $\max_{\mathbf{x}\in\mathcal{M}_{\{n\}},j\in\widetilde{\operatorname{BR}}(\mathbf{x})}u^{L}(\mathbf{x},j)$, respectively, which correspond to the right and left sides of Definition 4 (with $S=\{n\}$). Hence, given Definition 4, Property (a) can be achieved when the hyperplane is placed sufficiently close to the top. The additional requirement that a cover is proper is useful as we do not want any strategies in $C$ to induce $n$, which will eventually alter the boundary of $\widetilde{\operatorname{BR}}^{-1}(n)$. It turns out that this requirement is actually not strictly more demanding: according to Definition 4, any cover can be efficiently converted into a proper cover. Therefore, in the remainder of the paper, we simply refer to a proper cover as a cover. Moreover, a simple characterization of a cover is given in Definition 4: the existence of a cover requires a gap between $M_{S}$ and the maximin value $M_{[n]}$.

{restatable}

lemmalmmpropercover

Given a cover of set $S\subseteq[n]$, a proper cover of $S$ can be constructed in polynomial time.

{restatable}

lemmalmmcoveriff

For any $S\subseteq[n]$, a cover of $S$ exists if and only if $M_{S}>M_{[n]}$.

Next, we first demonstrate in Section 4.1 that given a cover of $\{n\}$ a set of strategies $\mathbf{x}^{1},\dots,\mathbf{x}^{m_{1}-1}$ satisfying the properties in 4.1 can be computed efficiently, and hence we learn $\mathbf{a}_{j}$. In the special case where $\{n\}$ does not admit a cover, Definition 4 implies that $M_{\{n\}}=M_{[n]}$. This means that $J=\{n\}$ and an arbitrary $\mathbf{x}^{*}\in\mathcal{M}_{\{n\}}$ already form a tuple satisfying (7), so by Lemma 3.3, we are done without learning $\mathbf{a}_{n}$. We demonstrate how to compute (or decide the existence of) a cover in Section 4.2. In summary, the approach to learning $\mathbf{a}_{n}$ is given in Figure 3.

4.1 Computing $\mathbf{x}^{1},\dots,\mathbf{x}^{m_{1}-1}$

We now describe how to compute $\mathbf{x}^{1},\dots,\mathbf{x}^{m_{1}-1}$, given a cover $\mu$ of $\{n\}$. For each $i\in[m_{1}-1]$, define the following set

[TABLE]

which is a facet of $\Delta_{m}$ that contains leader strategies satisfying Property (c) stated in 4.1. We aim to construct a function $\tilde{u}^{F}$ that induces a strategy $\mathbf{x}^{i}\in\Gamma_{i}$ to form an SSE with $n$, for each $i\in[m_{1}-1]$. This requires that the leader’s maximum attainable payoff with respect to $\tilde{u}^{F}$ is achievable at every $\Gamma_{i}$. If $\mathcal{M}_{\{n\}}$ is covered by only one action of the follower, say action $i$, this is relatively easy to achieve because the boundary separating $\widetilde{\operatorname{BR}}^{-1}(n)$ and $\widetilde{\operatorname{BR}}^{-1}(i)$ will be a hyperplane as in Figure 2. However, if more than one action is used to cover $\mathcal{M}_{\{n\}}$, the shape of the separation surface may become irregular, possibly with vertices sticking out in its interior. We need a more sophisticated construction to ensure that these interior points do not yield higher payoffs than the best leader strategies in $\Gamma_{i}$.

Our construction proceeds as follows, where we let $\mathbf{x}^{1},\dots,\mathbf{x}^{m_{1}-1}$ be parameterized by a vector $\mathbf{g}=(g_{1},\dots,g_{m_{1}-1})\in\mathbb{R}^{m_{1}-1}$. The construction ensures that one optimal strategy of the leader always appears in some $\Gamma_{i}$, and with fine tuning of $\mathbf{g}$ this can further be guaranteed for all $\Gamma_{i}$. {boxedtext}

Given $\mathbf{g}=(g_{1},\dots,g_{m_{1}-1})$, let

[TABLE]

where $W\coloneqq\min_{i,j}\mu(i,j)-1$.

• 2.

For every $i\in[m_{1}-1]$, pick an arbitrary leader strategy $\mathbf{x}^{i}$ such that

[TABLE]

where $f_{\mathbf{g}}:\Delta_{m}\to 2^{[n]}$ denotes the BR-correspondence of $\tilde{u}^{F}_{\mathbf{g}}$.

For simplicity, we omit the dependencies of $\mathbf{x}^{i}$ on $\mathbf{g}$ in the notation. We shall show soon that the arbitrary choice of $\mathbf{x}^{i}$ in (12) suffices for our purpose. Figure 4 provides an illustration of the notions defined above.

We show how to find an appropriate $\mathbf{g}$, so that $\mathbf{x}^{1},\dots,\mathbf{x}^{m_{1}-1}$ selected above satisfy the properties in 4.1. Indeed, the following lemma indicates that Properties (b) and (c) hold as long as we choose $\mathbf{g}$ with $g_{i}>0$ for all $i$.

{restatable}

lemmalmmxigt

$x_{i}^{i}>0$ for any choice of $\mathbf{g}$. Moreover, if $g_{i}>0$ then $\Gamma_{i}\cap f_{\mathbf{g}}^{-1}(n)\neq\emptyset$.

Proof.

For any $\mathbf{x}\in\Gamma_{i}$, if $x_{i}=0$ then $\mathbf{x}\notin f_{\mathbf{g}}^{-1}(n)$ because $\tilde{u}_{\mathbf{g}}^{F}$ is a cover of $\{n\}$. Moreover, if $g_{i}>0$, then consider a special $\mathbf{x}\in\Gamma_{i}$ such that $x_{i}=1$ and $x_{k}=0$ for all $k\in[n]\setminus\{i\}$. It can be verified that $\mathbf{x}\in f_{\mathbf{g}}^{-1}(n)$. ∎

Figure 4 shows that the leader’s utility on $(\mathbf{x},n)$ for $\mathbf{x}\in\Gamma_{i}$ only depends on the value of $x_{i}$.

{restatable}

lemmalmmminximaxuL

For all $\mathbf{x}\in\Gamma_{i}$, it holds that $u^{L}(\mathbf{x},n)=c\cdot x_{i}+d$ for some constants $c<0$ and $d$.

Proof.

Since $\mathbf{x}\in\Gamma_{i}$, we have $x_{k}=0$ for all $k=[m_{1}-1]\setminus\{i\}$, and hence $u^{L}(\mathbf{x},n)=\sum_{k=m_{1}}^{m}u^{L}(k,n)\cdot x_{k}+u^{L}(i,n)\cdot x_{i}$. Replacing $u^{L}(k,n)$ with $M_{\{n\}}$ for all $k=m_{1},\dots,m$ (by 4.1) and rearranging the terms give the desired result. ∎

To further ensure Property (a), we define the following set for every $\mathbf{g}$ with $g_{i}>0$ for all $i$:

[TABLE]

Clearly, given $\mathbf{x}^{1},\dots,\mathbf{x}^{m_{1}-1}$, $I_{\mathbf{g}}$ can be computed using oracle $\mathcal{A}_{\textnormal{SSE}}$. Moreover, Figure 4 implies that $\operatorname*{argmin}_{\mathbf{x}\in\Gamma_{i}\cap f_{\mathbf{g}}^{-1}(n)}x_{i}=\operatorname*{argmax}_{\mathbf{x}\in\Gamma_{i}\cap f_{\mathbf{g}}^{-1}(n)}u^{L}(\mathbf{x},n)$, so $I_{\mathbf{g}}$ is well-defined—it is independent of the specific choice of $\mathbf{x}^{i}$ in (12). We then aim to find $\mathbf{g}$ such that $I_{\mathbf{g}}=[m_{1}-1]$, so that Property (a) holds. Figure 4 implies that this can be achieved in polynomial time by inductively applying this result, thus leading to an efficient algorithm for computing $\mathbf{x}^{1},\dots,\mathbf{x}^{m_{1}-1}$.

{restatable}

theoremthmxxi

Suppose that $\mathbf{g}\in\mathbb{Q}_{>0}^{m_{1}-1}$. If $I_{\mathbf{g}}\neq[m_{1}-1]$, then there exists $\mathbf{g}^{\prime}\in\mathbb{Q}_{>0}^{m_{1}-1}$, such that $I_{\mathbf{g}}\subsetneq I_{\mathbf{g}^{\prime}}$. Moreover, $\mathbf{g}^{\prime}$ can be computed in time polynomial in the bit-size of $\mathbf{g}$.

Proof sketch.

To find $\mathbf{g}^{\prime}$, our approach is to increase $g_{i}$ if $(\mathbf{x}^{i},n)$ is not yet an SSE. Intuitively, increasing each $g_{i}$ will cause $u^{L}(\mathbf{x}^{i},n)$ to increase, so the hope is that when $u^{L}(\mathbf{x}^{i},n)$ is sufficiently large, $(\mathbf{x}^{i},n)$ becomes an SSE. Attention needs to be paid to the possibility that increasing $g_{i}$ might also cause $\max_{\mathbf{x}\in f^{-1}(n)\setminus(\cup_{i\in[m_{1}-1]}\Gamma_{i})}u^{L}(\mathbf{x},n)$ to increase even faster for strategies not in any $\Gamma_{i}$, so $(\mathbf{x}^{i},n)$ never outperform some $(\mathbf{x},n)$ not in $\bigcup_{i\in[m_{1}-1]}\Gamma_{i}$. Thanks to the way $\tilde{u}^{F}_{\mathbf{g}}$ is designed, this possibility can be eliminated by Section 4.1. It indicates that $\mathbf{x}^{1},\dots,\mathbf{x}^{m_{1}-1}$ are representatives of the leader’s best choice in $f_{\mathbf{g}}^{-1}(n)$.

The proof is then broken down into the following two cases.

• •

Case 1. If $I_{\mathbf{g}}=\emptyset$, we show that by increasing all $g_{i}$ to a sufficiently large number $N$, some $(\mathbf{x}^{i},n)$ will become an SSE. Moreover, $N$ can be bounded from above by a polynomial in the input size, so we can find a vector $\mathbf{g}^{\prime}$ such that $g^{\prime}_{i}\geq N$ for all $i$ in polynomial time. See Section B.3.

• •

Case 2. If $I_{\mathbf{g}}\neq\emptyset$, we show that by increasing $g_{i}$ for an arbitrary $i\notin I_{\mathbf{g}}$ to an appropriate number $g^{\prime}_{i}$ (while fixing $g_{k}$ for all $k\neq i$), the strategy profile $(\mathbf{x}^{i},n)$ will become a new SSE in addition to the existing ones. Moreover, $g^{\prime}_{i}$ can be computed in polynomial time. See Section B.3. ∎

{restatable}

lemmalemmaboundarypoint

For any $\mathbf{g}\in\mathbb{R}_{>0}^{m_{1}-1}$, it holds that $\max_{\mathbf{x}\in f_{\mathbf{g}}^{-1}(n)}u^{L}(\mathbf{x},n)=\max_{i\in[m_{1}-1]}u^{L}({\mathbf{x}}^{i},n)$.

4.2 Computing a Cover of $\{n\}$

Now we describe how to compute a cover of $\{n\}$ to complete this section. We first deal with an easy case, where $n\notin S^{*}\coloneqq\operatorname*{argmin}_{j\in[n]}M_{\{j\}}$.555Recall that we can efficiently decide whether $n\in S^{*}$ or not using $\mathcal{A}_{\textnormal{SSE}}$: to compare $M_{\{j\}}$ and $M_{\{j^{\prime}\}}$, pick $\mathbf{x}\in\mathcal{M}_{\{j^{\prime}\}}$ according to 3.1 and compare it with $M_{\{j\}}$ according to 3.2. In this case, any arbitrary $\ell\in S^{*}$ gives

[TABLE]

for all $\mathbf{x}\in\Delta_{m}$. Hence, any payoff matrix in which $\ell$ is a strictly dominant strategy forms a cover of $\{n\}$. Using our result in Section 4.1 and 4.1, we can then compute $\mathbf{a}_{n}$. By relabeling $n$ to $j$, this approach can also be used to find a cover of $\{j\}$ and obtain $\mathbf{a}_{j}$ for any $j\notin S^{*}$. Hence, in what follows we can assume that $\mathbf{a}_{j}$ is given for all $j\in[n]\setminus S^{*}$.

Next, consider the case where $n\in S^{*}$. To deal with this case, we present a more general result, Theorem 4.2, which finds a cover for any $S\subseteq[n]$. This method will also be useful for our argument in the next sections, where we need to find a cover for size-2 subsets of $[n]$. To apply this method requires a base function that make the leader gain at most $M_{S}$, best responding only actions in $S$. See the following definition for details. According to this definition, we can simply use a payoff function in which $n$ is a strictly dominant strategy of the follower as the base function for $\{n\}$.

Definition 5** (Base function).**

A payoff function $\tilde{u}^{F}$ with BR-correspondence $\widetilde{\operatorname{BR}}$ is a base function for set $S\subseteq[n]$ if (1) $\widetilde{\operatorname{BR}}(\mathbf{x})\subseteq S$ for all $\mathbf{x}\in\Delta_{m}$, and (2) the leader’s SSE payoff in game $(u^{L},\tilde{u}^{F})$ is $M_{S}$.

Theorem 4.2**.**

Suppose that $S\subseteq[n]$ and let $Q=\{j\in[n]:M_{\{j\}}=M_{S}\}$. Moreover, we are given the following elements: $\mathbf{a}_{j}$ for all $j\in[n]\setminus(S\cup Q)$, a base function $\tilde{u}^{F}$ for $S$, and $\mathcal{M}_{S}$ (as a polytope defined by a set of linear constraints). Then, in polynomial time, we can either compute a cover of $S$ or decide correctly that $S$ does not admit a cover.

Proof Sketch.

We construct the following payoff function, {boxedtext} For every $\mathbf{x}\in\Delta_{m}$,

[TABLE]

where for each $j\in[n]\setminus S$, $(\mathbf{b}_{j},c_{j})$ is a hyperplane such that for any $\mathbf{x}\in\Delta_{m}$:

[TABLE]

Theorem 4.2 shows that we can efficiently check if $\mu$ is a cover of $S$, and if it is not, then $S$ does not admit any other cover, either. Intuitively, $\mu$ aims to bring down the leader’s maximum attainable payoff in $\mathcal{M}_{S}$ to below $M_{S}$, so it needs to avoid responses that lead to $u^{L}(\mathbf{x},j)\geq M_{S}$ when the leader plays $\mathbf{x}$. Eq. 14 ensures this and roughly speaking it creates a “quasi-zero-sum” game on actions $j\notin S$. (Ideally, we could just use $\mu=-u^{L}$ to fulfill this task if we had the full information of $u^{L}$.) It then remains to find a way to efficiently compute a set of hyperplanes $(\mathbf{b}_{j},c_{j})$ satisfying Eq. 14 to finish the construction of $\mu$, which is further demonstrated in the proof of Theorem 4.2 (deferred to Section B.6). ∎

{restatable}

lemmalmmfindcovermu

It can be decided in polynomial time whether the payoff matrix $\mu$ defined in Eq. 13 is a cover of $S$ or not. Moreover, if it is not a cover of $S$, then $S$ does not admit any other cover, either.

{restatable}

lemmalmmfindcoverbbjcj

A hyperplane $(\mathbf{b}_{j},c_{j})$ satisfying Eq. 14 can be computed in polynomial time for every $j\in[n]\setminus S$.

5 Learning $J$ and $\mathbf{x}^{*}$

We show how to learn $J$ and $\mathbf{x}^{*}$ in this section. Recall that as outlined in Figure 1, we want to find $J\subseteq[n]$ and $\mathbf{x}^{*}\in\Delta_{m}$, such that $u^{L}(\mathbf{x}^{*},j)=M_{J}=M_{[n]}$ for all $j\in J$. First, we highlight several observations that we will use throughout this section.

Observation 5.1**.**

The following assumptions are without loss of generality:

1. (a)

$\mathbf{a}_{j}$* is given for all $j\in[n]$, and $M_{\{j\}}>M_{[n]}$.* 2. (b)

$1\in\operatorname*{argmin}_{j\in[n]}M_{\{j\}}$. 3. (c)

$a_{1,j}\neq a_{1,k}$* for some $j,k\in[m]$.*

Specifically, Item 5.1(a) results directly from the learning outcome of Section 4. Item 5.1(b) holds by relabeling action $1$ and an arbitrary action in $\operatorname*{argmin}_{j\in[n]}M_{\{j\}}$. In the case where Item 5.1(c) does not hold, we have $a_{1,1}=a_{1,2}=\dots=a_{1,m}$; hence, $u^{L}(\mathbf{x},1)$ is a constant and $u^{L}(\mathbf{x},1)=M_{\{1\}}$ for all $\mathbf{x}\in\Delta_{m}$. By Item 5.1(a), it then follows that $u^{L}(\mathbf{x},1)>M_{[n]}$ for all $\mathbf{x}\in\Delta_{m}$. According to the desired property of $J$ defined in (7), this means that we can essentially exclude all actions that do not satisfy Item 5.1(c) from our consideration completely—we consider the residue game defined on the remaining action set of the follower after excluding these actions.

In addition to the assumptions, when $\mathbf{a}_{j}$ is known, we can also assume access to the following oracle $\mathcal{A}_{\textnormal{ER}}$ (Definition 6), which determines if an action $j$ is an SSE response. Specifically, to determine whether $j$ is an SSE response of game $(u^{L},\tilde{u}^{F})$, it suffices to check whether $(\mathbf{x},j)$ is an SSE (by using $\mathcal{A}_{\textnormal{SSE}}$) for an arbitrary $\mathbf{x}$ in the set $\operatorname*{argmax}_{\mathbf{x}^{\prime}\in\widetilde{\operatorname{BR}}^{-1}(j)}u^{L}(\mathbf{x}^{\prime},j)=\operatorname*{argmax}_{\mathbf{x}^{\prime}\in\widetilde{\operatorname{BR}}^{-1}(j)}\mathbf{a}_{j}\cdot\mathbf{x}^{\prime}$, so $\mathbf{x}$ can be computed efficiently when $\mathbf{a}_{j}$ is known.

Definition 6** (Equilibrium response oracle).**

Given a game $\widetilde{\mathcal{G}}=(u^{L},\tilde{u}^{F})$ and an action $j\in[n]$ of the follower, the equilibrium response (ER) oracle, denoted $\mathcal{A}_{\textnormal{ER}}$, outputs whether $j$ is an SSE response of $\widetilde{\mathcal{G}}$.

To learn $J$ and $\mathbf{x}^{*}$, we will use the following strategically equivalent payoff matrix $\tilde{u}^{L}$ of the leader, as a surrogate for the original matrix $u^{L}$:

[TABLE]

where $W=1+\max_{i\in[m],\ell\in\widehat{J}}\tilde{u}^{L}(i,\ell)$ (a number that is sufficiently large); $\gamma_{j}$ and $\beta_{j}$ are the parameters that give $u^{L}(\mathbf{x},j)\equiv\gamma_{j}\cdot\mathbf{a}_{j}\cdot\mathbf{x}+\beta_{j}$; and

[TABLE]

According to Lemma 6, we can compute $J$ and $\mathbf{x}^{*}$ based on $\tilde{u}^{L}$ instead of the original unknown matrix $u^{L}$, where we use the notation

[TABLE]

for all $S\subseteq[n]$, defined analogously to $M_{S}$ (Eq. 4). The problem is then equivalent to computing the maximin strategy of $\tilde{u}^{L}$, which is tractable if all the parameters defining $\tilde{u}^{L}$ are known.

{restatable}

lemmalmmtildeuLJ

For any $J\subseteq[n]$ and $\mathbf{x}^{*}\in\Delta_{m}$, Eq. 7 holds if and only if $\tilde{u}^{L}(\mathbf{x}^{*},j)=\widetilde{M}_{J}=\widetilde{M}_{[n]}$ for all $j\in J$.

It remains to compute the parameters needed for constructing $\tilde{u}^{L}$, that is, $\widehat{J}$, $\gamma_{j}$, and $\beta_{j}$. Indeed, $\widehat{J}$ can be efficiently computed by comparing $u^{L}(i,j)$ with $M_{\{1\}}=\max_{i\in[n]}u^{L}(i,1)$ for all $i,j$; these relations are known according to 3.2. Also note that $\widehat{J}\neq\emptyset$, since otherwise we would have $M_{[n]}=M_{\{1\}}$, which contradicts Item 5.1(a).

As for $\gamma_{j}$ and $\beta_{j}$, we will not learn them directly, but only learn the quantities $\gamma_{j}/\gamma_{1}$ and $(\beta_{j}-\beta_{1})/\gamma_{1}$. To this end, we will find two different pairs of strategies $\mathbf{x}_{1},\mathbf{y}_{1}$ and $\mathbf{x}_{2},\mathbf{y}_{2}$, such that $u^{L}(\mathbf{x}_{1},1)=u^{L}(\mathbf{y}_{1},j)$ and $u^{L}(\mathbf{x}_{2},1)=u^{L}(\mathbf{y}_{2},j)$. This gives the following system of linear equations:

[TABLE]

Rearranging the terms, it is easy to see that when $\mathbf{x}_{1}$, $\mathbf{x}_{2}$, $\mathbf{y}_{1}$, and $\mathbf{y}_{2}$ are given (in addition to $\mathbf{a}_{1}$ and $\mathbf{a}_{j}$, which we already know), the above is a system of equations about $\gamma_{j}/\gamma_{1}$ and $(\beta_{j}-\beta_{1})/\gamma_{1}$; hence, solving the system gives the two desired quantities. We hereafter call each of the two pairs $\mathbf{x}_{1},\mathbf{y}_{1}$ and $\mathbf{x}_{2},\mathbf{y}_{2}$ a reference pair. We in particular look for two reference pairs that make the above equation system non-degenerate, so that it has a unique solution.

As a final remark, the reason why we use $\widehat{J}$ instead of $[n]$ is that $\gamma_{j}/\gamma_{1}$ and $(\beta_{j}-\beta_{1})/\gamma_{1}$ cannot be learned for actions $j\notin\widehat{J}$. Indeed, the leader’s payoffs for actions not in $\widehat{J}$ is too high, so these actions will not contribute to making the set $J$ such that $M_{J}=M_{[n]}$.

An overview of the approach to learning $J$ and $\mathbf{x}^{*}$ is provided in Figure 5.

5.1 Finding the First Reference Pair

For ease of description, in what follows, we assume $2\in\widehat{J}$ and present how to find the reference pairs for computing $\gamma_{2}/\gamma_{1}$ and $(\beta_{2}-\beta_{1})/\gamma_{1}$. The results generalize to any $j\in\widehat{J}$ by relabeling $2$ to $j$.

Observation 5.2**.**

Without loss of generality, we can assume that $2\in\widehat{J}$.

Next we show how to find a first reference pair. Recall that we want to find a pair $\mathbf{x},\mathbf{y}$ with $u^{L}(\mathbf{x},1)=u^{L}(\mathbf{y},2)$. We in particular aim to find a pair such that

[TABLE]

We use the following payoff function $\tilde{u}^{F}_{d}$ parameterized by a number $d\in\mathbb{R}$. {boxedtext} For every $\mathbf{x}\in\Delta_{m}$, let

[TABLE]

Let $f_{d}$ be the BR-correspondence defined by $\tilde{u}^{F}_{d}$. It can be verified that

[TABLE]

as illustrated in Figure 6. Let $\mathcal{G}_{d}\coloneqq(u^{L},\tilde{u}^{F}_{d})$ be the game induced by $\tilde{u}^{F}_{d}$.

We then search for a $d$ that makes both actions $1$ and $2$ SSE responses. Intuitively, when $d$ is sufficiently small, the follower responding according to $f_{d}$ leads to action $1$ being the sole best response against the entire strategy space $\Delta_{m}$ of the leader and hence being the only SSE response of $\mathcal{G}_{d}$. Conversely, when $d$ is sufficiently large, action $2$ becomes the only SSE response of $\mathcal{G}_{d}$. Therefore, the goal is to identify a point between these two extremes, where both actions $1$ and $2$ are SSE responses. This point is exactly

[TABLE]

(Figure 6). Note that, we cannot compute $d^{*}$ directly using the above equation since we do not know any of the values $M_{\{1,2\}}$, $\beta_{2}$, and $\gamma_{2}$. Instead, we will use binary search to find it out: according to Figure 6 presented below, $\mathcal{G}_{d}$ has different SSE responses when $d<d^{*}$ and $d>d^{*}$, so using the SSE oracle, we can identify whether a candidate value is smaller or larger than $d^{*}$.

{restatable}

lemmalmmfdmain

Action $1$ is an SSE response of $\mathcal{G}_{d}$ if and only if $d\leq d^{*}$, and action $2$ is an SSE response of $\mathcal{G}_{d}$ if and only if $d\geq d^{*}$.

Once $d^{*}$ is identified, we can compute the first reference pair immediately. Intuitively, both actions $1$ and $2$ are SSE responses of $\mathcal{G}_{d^{*}}=(u^{L},\tilde{u}_{d^{*}}^{F})$. So picking arbitrary $\mathbf{x}\in\operatorname*{argmax}_{\mathbf{x}^{\prime}\in f_{d^{*}}^{-1}(1)}u^{L}(\mathbf{x}^{\prime},1)$ and $\mathbf{y}\in\operatorname*{argmax}_{\mathbf{y}^{\prime}\in f_{d^{*}}^{-1}(2)}u^{L}(\mathbf{y}^{\prime},2)$ gives $u^{L}(\mathbf{x},1)=u^{L}(\mathbf{y},2)$, as desired. The computation of $\mathbf{x}$ (and similarly $\mathbf{y}$) can be handled by solving, equivalently, $\max_{\mathbf{x}^{\prime}\in f_{d^{*}}^{-1}(1)}\mathbf{a}_{1}\cdot\mathbf{x}^{\prime}$, where the constraint $\mathbf{x}^{\prime}\in f_{d^{*}}^{-1}(1)$ is further equivalent to $\mathbf{a}_{2}\cdot\mathbf{x}^{\prime}\geq d^{*}$. Hence, the task reduces to solving an LP, which can be done in polynomial time. The following theorem presents a stronger result saying that the strategies in this reference pair yields the maximin payoff of ${\{1,2\}}$ for the leader.

{restatable}

theoremthmcomputedstar

A reference pair $\mathbf{x},\mathbf{y}\in\Delta_{m}$ such that $u^{L}(\mathbf{x},1)=u^{L}(\mathbf{y},2)=M_{\{1,2\}}$ can be computed in polynomial time.

5.2 Finding the Second Reference Pair

Next, we search for a second reference pair. We aim to find a pair $\mathbf{x},\mathbf{y}$ such that

[TABLE]

Given ($\star$ ‣ 5.1), the requirement that the payoffs are strictly smaller than $M_{\{1,2\}}$ ensures that (17) has a unique solution. Similar to our approach to finding the first reference pair, we aim to construct a BR-correspondence to induce two SSEs $(\mathbf{x},1)$ and $(\mathbf{y},2)$.

As illustrated in Figure 7, the high-level idea is to pull back the boundaries of the best-response regions $f_{d^{*}}^{-1}(1)$ and $f_{d^{*}}^{-1}(2)$ (where we add a new boundary $\mathbf{a}_{1}\cdot\mathbf{x}=d_{1}$ to the region corresponding to action $1$). By pulling back the boundaries by appropriate distances, we can keep the leader’s maximum attainable payoffs in these two regions equal, while at the same time they become strictly smaller than $M_{\{1,2\}}$; hence, we obtain a pair satisfying ($\star\star$ ‣ 5.2).

Hence, the key is to maintain both actions $1$ and $2$ as SSE responses. The contraction of the best-response regions of these two actions means that a blank region will appear, which needs to be allocated to some response of the follower for the BR-correspondence to be well-defined. In particular, we need a follower action that gives the leader a sufficiently low payoff, so that actions $1$ and $2$ remain to be SSE responses. Sometimes we cannot find a single action of the follower to fulfill this task so in general we need a cover of ${\{1,2\}}$ which may involve multiple actions.

Our main result is stated in Figure 7. To better illustrate the approach, we will first present an algorithm that uses BR-correspondence queries, queries that use games in the form $\widetilde{\mathcal{G}}=(u^{L},\widetilde{\operatorname{BR}})$, where $\widetilde{\operatorname{BR}}:\Delta_{m}\to 2^{[n]}$ is a BR-correspondence. We assume temporarily that $\mathcal{A}_{\textnormal{SSE}}$ can handle such queries. Ideally, the BR-correspondences used should also be realized by valid payoff matrices, but this is not always the case with the algorithm presented next. Hence, we also design a stronger algorithm that always uses payoff matrices to query the SSE oracle. The process is much more involved but can be better understood based on intuition conveyed from the former, and we leave the details to Section E.4.

{restatable}

theoremthmsecondrefpayoffquery

A reference pair $\mathbf{x},\mathbf{y}\in\Delta_{m}$ such that $u^{L}(\mathbf{x},1)=u^{L}(\mathbf{y},2)<M_{\{1,2\}}$ can be computed in polynomial time.

5.2.1 Querying by Using BR-correspondences

Define the following partition $P=(P_{0},P_{1},P_{2})$ of $\Delta_{m}$, which is parameterized by two numbers $d_{1}$ and $d_{2}$:

[TABLE]

For ease of description, we will sometimes omit the dependencies on $d_{1}$ and $d_{2}$ and just write $P_{0}$, $P_{1}$, and $P_{2}$ when the parameters are clear from the context. Based on the partition, we define the following BR-correspondence $f_{d_{1},d_{2}}$, which generalizes the BR-correspondence $f_{d}$ defined in the previous section (see (18)). {boxedtext} Let $f_{d_{1},d_{2}}:\Delta_{m}\to 2^{[n]}$ be a BR-correspondence such that for every $\mathbf{x}\in\Delta_{m}$:

[TABLE]

where $h:\Delta_{m}\to 2^{[n]\setminus{\{1,2\}}}$ is the BR-correspondence of an arbitrary (proper) cover of $\{1,2\}$.

Hence, according to this construction, we have

[TABLE]

Moreover, compared with $f_{d}$, we have $P_{2}(d_{1},d_{2})=f_{d_{2}}^{-1}(2)$, and $P_{1}(d_{1},d_{2})\cup P_{0}(d_{1},d_{2})=f_{d_{2}}^{-1}(1)$. Figure 7 illustrates the structure of $f_{d_{1},d_{2}}$.

Our goal is to find two numbers $d_{1}$ and $d_{2}$ such that both actions $1$ and $2$ are SSE responses of $\mathcal{G}_{d_{1},d_{2}}\coloneqq(u^{L},f_{d_{1},d_{2}})$. Before we present our algorithm for computing $d_{1}$ and $d_{2}$, we define several other useful notions and make a few observations about $f_{d_{1},d_{2}}$.

We define

[TABLE]

Moreover, let

[TABLE]

for $j\in{\{1,2\}}$ (i.e., $d_{2}^{*}=d^{*}$ in (19)). It can be verified that when $d_{1}=d_{1}^{*}$ and $d_{2}=d_{2}^{*}$, $P$ coincides with $f_{d^{*}}$. Moreover, we have:

• •

$P_{2}(d_{1}^{*},d_{2}^{*})=f_{d^{*}}^{-1}(2)\neq\emptyset$, $P_{1}(d_{1}^{*},d_{2}^{*})=f_{d^{*}}^{-1}(1)\neq\emptyset$, and $P_{0}(d_{1}^{*},d_{2}^{*})=\mathcal{M}_{{\{1,2\}}}$; and

• •

for all $\odot\in\{>,<,=\}$ and $j\in{\{1,2\}}$, $u^{L}(\mathbf{x},j)\odot M_{\{1,2\}}$ if and only if $\mathbf{a}_{j}\cdot\mathbf{x}\odot d_{j}^{*}$.

These two values $d_{1}^{*}$ and $d_{2}^{*}$ will be useful for our algorithm. Indeed, we search for the two numbers $d_{1}$ and $d_{2}$ we aim to find in the domains $(-\infty,d_{1}^{*})$ and $(-\infty,d_{2}^{*})$, respectively. We require that $d_{1}<d_{1}^{*}$ and $d_{2}<d_{2}^{*}$ in order to avoid the trivial solution $d_{1}=d_{1}^{*}$ and $d_{2}=d_{2}^{*}$, which would yield the same pair as the first reference pair we obtained.

Existence and Computation of $h$

To obtain $f_{d_{1},d_{2}}$ requires computing a cover $h$ of ${\{1,2\}}$. We directly invoke Theorem 4.2 presented earlier to accomplish this task. Now that $\mathbf{a}_{j}$ is known for all $j$, to apply this theorem, we only need to supply with a base function for ${\{1,2\}}$, as well as $\mathcal{M}_{\{1,2\}}$. Indeed, the function $\tilde{u}_{d^{*}}^{F}$ we obtained in the previous section and the region $P_{0}(d_{1}^{*},d_{2}^{*})$ defined above fulfills these demands. In the special case where ${\{1,2\}}$ does not admit a cover, Lemma E.1 in the appendix demonstrates that a pair of satisfying $J$ and $\mathbf{x}^{*}$ can be obtained without a second reference pair.

Pinning Down $d_{1}$ and $d_{2}$

To find $d_{1}$ and $d_{2}$, we first restrict our search in a one-dimensional space: we aim to find a value $\epsilon>0$ such that at least one of actions $1$ and $2$ are SSE responses of $\mathcal{G}_{d_{1}^{*}-\epsilon,d_{2}^{*}-\epsilon}$. We present the following lemma.

{restatable}

lemmalmmGdeps

There exists $\hat{\epsilon}>0$, such that for all $d_{1}\in[d_{1}^{*}-\hat{\epsilon},d_{1}^{*}]$ and $d_{2}\in[d_{2}^{*}-\hat{\epsilon},d_{2}^{*}]$:

• (i)

at least one of $j\in{\{1,2\}}$ is an SSE response of $\mathcal{G}_{d_{1},d_{2}}$; and

• (ii)

$V_{d_{1},d_{2}}^{L}(j)=\gamma_{j}\cdot d_{j}+\beta_{j}$ for $j\in{\{1,2\}}$.

Moreover, assuming $\mathcal{A}_{\textnormal{SSE}}$ can handle BR-correspondence queries, $\hat{\epsilon}$ is computable in polynomial time.

The proof of Section 5.2.1 is deferred to Section E.2. Using Section 5.2.1, we obtain $d_{1}=d_{1}^{*}-\hat{\epsilon}$ and $d_{2}=d_{2}^{*}-\hat{\epsilon}$, such that $\mathcal{G}_{d_{1},d_{2}}$ has at least one of actions $1$ and $2$ as an SSE response. If it happens that both actions $1$ and $2$ are SSE responses, we are done with a second reference pair. Otherwise, e.g., suppose that action $2$ is not an SSE response, it means that we need to expand $P_{2}$ to increase the leader’s maximum attainable payoff for strategies in this region; we do this simply by increasing $d_{2}$, and we search for a value of $d_{2}$ that makes both actions $1$ and $2$ an SSE response. This leads to Section 5.2.1 as a weaker version of Figure 7. The detailed proofs of Sections 5.2.1 and 7 can be found in the appendix.

{restatable}

theoremthmsecondrefpairBRcorrespondence

Assume that $\mathcal{A}_{\textnormal{SSE}}$ can handle BR-correspondence queries. A reference pair $\mathbf{x},\mathbf{y}\in\Delta_{m}$ such that $u^{L}(\mathbf{x},1)=u^{L}(\mathbf{y},2)<M_{\{1,2\}}$ can be computed in polynomial time.

6 Conclusion

Whereas the polynomial-time tractability of the follower’s optimal manipulation in Birmpas et al. [2021] assumes full information on the leader’s payoffs, we showed that such strict information advantage is not quite necessary: in the learning setting, polynomial number of information exchanges with the leader suffices for the follower to find out the optimal payoff function to report.

Our result reveals an interesting difference in the effects of information asymmetry on the two players in a Stackelberg game and points out potential risks of applying strategy commitment under a lack of information: while a leader needs an exponential number of queries to learn a follower’s best response correspondence in the worst case [Peng et al., 2019], it only takes the follower a polynomial number of queries to collect enough payoff information of the leader and achieve optimal deception. Thus, special care is needed, when one wants to apply strategy commitment to gain advantage in games but without sufficient information.

We note that despite of the SSE oracle defined in our paper, there can be other definitions of oracles, possibly weaker than ours. For example, it may be worth studying oracles that only take a fake payoff function as input and output an SSE according to some underlying tie-breaking rules. This might be more realistic as it is the leader who chooses which equilibrium strategy to play. However, our results do not directly extend to this setting. One insight into this is that, under the weaker oracle, we may lose the ability to check whether the leader utilities under two strategy profiles are equal. Hence, we may not be able to learn the precise quantities necessary for our algorithms. We leave it as an interesting open problem whether there can be an optimal, or a near-optimal algorithm for the follower to learn to deceive when they have access to such weaker oracles.

Appendix A Proof of Lemma 3.3

\thmequgame

Proof.

Given Lemma 3.3, LP (5) reduces to deciding the inducibility of any given strategy profile $(\mathbf{y},k)$. We demonstrate that this inducibility problem can be solved in polynomial time. Specifically, we will argue that the following algorithm produces an inducibility witness $\tilde{u}^{F}$ for $(\mathbf{y},k)$ in polynomial time: that is, $(\mathbf{y},k)$ is inducible (with respect to $u^{L}$) if and only if it is an SSE in $(u^{L},\tilde{u}^{F})$. Hence, by querying $\mathcal{A}_{\textnormal{SSE}}$ to check whether $(\mathbf{y},k)$ is an SSE of $(u^{L},\tilde{u}^{F})$, we can decide the inducibility of $(\mathbf{y},k)$. The stated result then follows.

{boxedtext}

Construct an inducibility witness of $(\mathbf{y},k)$:

For every $j$ such that $M_{\{j\}}=M_{[n]}$, since $\mathbf{a}_{j}$ is not given, let $\mathbf{a}_{j}$ be a vector in $\mathbb{R}^{m}$ such that: $a_{ji}=1$ if $i\in\mathcal{M}_{\{j\}}$ and $a_{ji}=0$ otherwise.

• 2.

Construct a payoff matrix $\tilde{u}^{L}\in\mathbb{R}^{m\times n}$ corresponding to the following payoff function:

[TABLE]

• 3.

Decide if $(\mathbf{y},k)$ is inducible with respect to $\tilde{u}^{L}$:

• –

If it is inducible, compute a payoff matrix $\tilde{u}^{F}\in\mathbb{R}^{m\times n}$ such that $(\mathbf{y},k)$ is an SSE in $(\tilde{u}^{L},\tilde{u}^{F})$;

• –

Otherwise, pick an arbitrary $\tilde{u}^{F}\in\mathbb{R}^{m\times n}$.

• 4.

Output $\tilde{u}^{F}$.

Namely, the above algorithm constructs a “surrogate” matrix $\tilde{u}^{L}$ using the given information $\mathbf{a}_{j}$, $J$, and $\mathbf{x}^{*}$. Then a witness $\tilde{u}^{F}$ is produced using $\tilde{u}^{L}$. The polynomial run-time of the algorithm is readily seen. In particular, Step 3 can be done in polynomial time according to Theorem 2.1. (Note that all parameters of $\tilde{u}^{L}$ is known, so this is a full-information setting.) We next prove that $(\mathbf{y},k)$ is inducible if and only if it is an SSE in $(u^{L},\tilde{u}^{F})$, where $\tilde{u}^{F}$ is the matrix produced by the above algorithm.

Indeed, the “if” direction is trivial, so we prove that if $(\mathbf{y},k)$ is inducible then it must be an SSE in $(u^{L},\tilde{u}^{F})$. Suppose that $(\mathbf{y},k)$ is inducible in the sequel. According to Theorem 2.1, it must be that $u^{L}(\mathbf{y},k)\geq M_{[n]}$ in this case.

In what follows, we first argue that the following statements hold for any $\mathbf{x}\in\Delta_{m}$ (1).

[TABLE]

With this result, we can then prove 2 and 3 to complete the proof: 2 implies that Step 3 of the above algorithm outputs a $\tilde{u}^{F}$ that makes $(\mathbf{y},k)$ an SSE of $(\tilde{u}^{L},\tilde{u}^{F})$, instead of an arbitrary matrix. Given this, 3 further confirms that $(\mathbf{y},k)$ is also an SSE of $(u^{L},\tilde{u}^{F})$.

Claim 1**.**

(26) and (27) hold for any $\mathbf{x}\in\Delta_{m}$.

Proof.

Note that $\tilde{u}^{L}(\mathbf{x},j)>0\Longleftrightarrow\mathbf{a}_{j}\cdot\mathbf{x}>\mathbf{a}_{j}\cdot\mathbf{x}^{*}$ and $\tilde{u}^{L}(\mathbf{x},k)>0\Longleftrightarrow\mathbf{a}_{k}\cdot\mathbf{x}>\mathbf{a}_{k}\cdot\mathbf{y}$ follow directly by (8), so it remains to prove the second part of each statement. Moreover, if $M_{\{j\}}\neq M_{[n]}$ and $M_{\{k\}}\neq M_{[n]}$, $\mathbf{a}_{j}$ and $\mathbf{a}_{k}$ satisfy (6), so it is easy to see that the second part of each statement also holds by expanding $u^{L}$ according to (6).

Now consider the case where $M_{\{j\}}=M_{[n]}$. We have

[TABLE]

where $u^{L}(\mathbf{x}^{*},j)=M_{[n]}$ follows by the definition of $\mathbf{x}^{*}$ in (7) (and the assumption that $j\in J$), and $M_{\{j\}}\geq u^{L}(\mathbf{x},j)$ follows by the definition that $M_{\{j\}}=\max_{\mathbf{x}^{\prime}\in\Delta_{m}}u^{L}(\mathbf{x}^{\prime},j)$. At the same time, $u^{L}(\mathbf{x}^{*},j)=M_{\{j\}}$ also implies that $\mathbf{x}^{*}\in\mathcal{M}_{\{j\}}$. Hence, according to the way $\mathbf{a}_{j}$ is defined in Step 1 of the above algorithm, we have $\mathbf{x}^{*}\in\operatorname*{argmax}_{\mathbf{x}^{\prime}\in\Delta_{m}}\mathbf{a}_{j}\cdot\mathbf{x}^{\prime}\geq\mathbf{a}_{j}\cdot\mathbf{x}$. Consequently, both $\mathbf{a}_{j}\cdot\mathbf{x}>\mathbf{a}_{j}\cdot\mathbf{x}^{*}$ and $u^{L}(\mathbf{x},j)>u^{L}(\mathbf{x}^{*},j)$ are always false in this case, so (26) holds.

Finally, consider the case where $M_{\{k\}}=M_{[n]}$. In this case, we can establish $M_{\{k\}}\geq u^{L}(\mathbf{y},k)\geq M_{[n]}$. Consequently, $u^{L}(\mathbf{y},k)\geq M_{[n]}$, so we have $\mathbf{y}\in\mathcal{M}_{\{k\}}$. Similarly to the above case where $M_{\{j\}}=M_{[n]}$, we can show that both $\mathbf{a}_{k}\cdot\mathbf{x}>\mathbf{a}_{k}\cdot\mathbf{y}$ and $u^{L}(\mathbf{x},k)>u^{L}(\mathbf{y},k)$ must always be false in this case (i.e., by putting $\mathbf{y}$ in place of $\mathbf{x}^{*}$ and $k$ in place of $j$). Hence, (27) holds. ∎

Claim 2**.**

$(\mathbf{y},k)$* is inducible with respect to $\tilde{u}^{L}$ (assuming that $(\mathbf{y},k)$ is inducible with respect to $u^{L}$).*

Proof.

By construction, $\tilde{u}^{L}(\mathbf{y},k)=0$. Hence, by Theorem 2.1, it suffices to show that the maximin value of $\tilde{u}^{L}$ is at most [math], i.e., $\widetilde{M}_{[n]}\coloneqq\max_{\mathbf{x}\in\Delta_{m}}\min_{j\in[n]}\tilde{u}^{L}(\mathbf{x},j)\leq 0$.

Suppose for the sake of contradiction that $\widetilde{M}_{[n]}>0$. By definition, this means that there exists $\mathbf{x}\in\Delta_{m}$ such that $\tilde{u}^{L}(\mathbf{x},j)>0$ for all $j\in[n]$. Applying (26) and (27) gives $u^{L}(\mathbf{x},j)>u^{L}(\mathbf{x}^{*},j)=M_{[n]}$ for all $j\in J\setminus\{k\}$ (where $u^{L}(\mathbf{x}^{*},j)=M_{[n]}$ according to (7)), and $u^{L}(\mathbf{x},k)>u^{L}(\mathbf{y},k)\geq M_{[n]}$. It then follows that $\min_{j\in J}u^{L}(\mathbf{x},j)>M_{[n]}$, which contradicts the condition in the definition of $J$, i.e., $M_{J}=M_{[n]}$. ∎

Claim 3**.**

$(\mathbf{y},k)$* is an SSE of $(u^{L},\tilde{u}^{F})$ (assuming that it is an SSE of $(\tilde{u}^{L},\tilde{u}^{F})$).*

Proof.

Suppose that $(\mathbf{y},k)$ is an SSE in $(\tilde{u}^{L},\tilde{u}^{F})$. Let $\widetilde{\operatorname{BR}}$ be the BR-correspondence of $\tilde{u}^{F}$. Pick arbitrary $\mathbf{x}\in\Delta_{m}$ and $j\in\widetilde{\operatorname{BR}}(\mathbf{x})$. Since $(\mathbf{y},k)$ is an SSE in $(\tilde{u}^{L},\tilde{u}^{F})$, by definition, we have $\tilde{u}^{L}(\mathbf{x},j)\leq\tilde{u}^{L}(\mathbf{y},k)$. Substituting the right hand side with (8), we get that $\tilde{u}^{L}(\mathbf{x},j)\leq\mathbf{a}_{k}\cdot\mathbf{y}-\mathbf{a}_{k}\cdot\mathbf{y}=0$. According to (8), this means that $j\in J\cup\{k\}$.

If $j\in J\setminus\{k\}$, then applying (26), we get that

[TABLE]

(Recall that $u^{L}(\mathbf{y},k)\geq M_{[n]}$ as $(\mathbf{y},k)$ is inducible). If $j=k$, then applying (27), we get that

[TABLE]

Hence, $u^{L}(\mathbf{x},j)\leq u^{L}(\mathbf{y},k)$ holds in both cases. Since the choice of $\mathbf{x}$ and $j$ is arbitrary, we have

[TABLE]

It remains to show that $k\in\widetilde{\operatorname{BR}}(\mathbf{y})$. Indeed, according to 2 and Step 3 of the above algorithm, $(\mathbf{y},k)$ must be an SSE in $(\tilde{u}^{L},\tilde{u}^{F})$, which means that $k\in\widetilde{\operatorname{BR}}(\mathbf{y})$. ∎

The proof is complete. ∎

Appendix B Omitted Proofs in Section 4

B.1 Proof of Definition 4

\lmmpropercover

Proof.

Suppose that $\mu$ is a cover of $S$. Consider the following matrix $\mu^{\prime}$:

[TABLE]

where $W\coloneqq\min_{i,j\in[m]\times[n]\setminus S}\mu(i,j)-1$. We argue that $\mu^{\prime}$ is a proper cover of $S$. Indeed, it is proper since by the above construction no action in $S$ can be a best response to any $\mathbf{x}\in\Delta_{m}$. It suffices to prove that $\mu^{\prime}$ is a cover.

Let $f$ and $f^{\prime}$ be the BR-correspondences of $\mu$ and $\mu^{\prime}$, respectively. Note that for all $\mathbf{x}\in\mathcal{M}_{S}$, it must be that $S\cap f(\mathbf{x})=\emptyset$. Indeed, if $k\in S\cap f(\mathbf{y})$ and $\mathbf{y}\in\mathcal{M}_{S}$, we would then have

[TABLE]

contradicting the assumption that $\mu$ is a cover.

By construction, $f^{\prime}(\mathbf{x})=f(\mathbf{x})$ if $S\cap f(\mathbf{x})=\emptyset$. Hence, $f^{\prime}(\mathbf{x})=f(\mathbf{x})$ for all $\mathbf{x}\in\mathcal{M}_{S}$. It follows that

[TABLE]

Hence, $\mu^{\prime}$ is a cover of $S$. ∎

B.2 Proof of Definition 4

\lmmcoveriff

Proof.

We first prove the necessity. Suppose a cover $\mu$ of $S$ exists. According to the definition of a cover, for any $\mathbf{x}\in\mathcal{M}_{S}$, there exists $j\in[n]\setminus S$, such that $u^{L}(\mathbf{x},j)<M_{S}$. Thus, for all $\mathbf{x}\in\mathcal{M}_{S}$, we have

[TABLE]

Meanwhile, for all $\mathbf{x}\notin\mathcal{M}_{S}$,

[TABLE]

Thus, $M_{[n]}=\max_{\mathbf{x}\in\Delta_{m}}\min_{j\in[n]}u^{L}(\mathbf{x},j)<M_{S}$.

Next, consider the sufficiency. Suppose that $M_{S}>M_{[n]}$. We show that a function $\mu$ such that $\mu(i,j)=-u^{L}(i,j)$ is a cover of $S$. Let $f$ be the BR-correspondence of $\mu$. By construction, $\operatorname*{argmax}_{k\in[n]}\mu(\mathbf{x},k)=\operatorname*{argmin}_{k\in[n]}u^{L}(\mathbf{x},k)$ for any $\mathbf{x}\in\Delta_{m}$, so $j\in f(\mathbf{x})$ implies that $u^{L}(\mathbf{x},j)=\min_{k\in[n]}u^{L}(\mathbf{x},k)$. It follows that

[TABLE]

for all $\mathbf{x}\in\Delta_{m}$. Thus, $\max_{\mathbf{x}\in\mathcal{M}_{S}}\min_{j\in f(\mathbf{x})}u^{L}(\mathbf{x},j)<M_{S}$, which completes the proof. ∎

B.3 Formal Proof of Figure 4

We first present the proof of Section 4.1, then the proof is completed by Section B.3 and Section B.3, which solves the cases of $I_{\mathbf{g}}=\emptyset$ and $I_{\mathbf{g}}\neq\emptyset$ respectively.

\lemmaboundarypoint

Proof.

We prove that every vertex $\mathbf{v}$ of $f_{\mathbf{g}}^{-1}(n)$ lies in some $\Gamma_{i}$, $i\in[m_{1}-1]$; in other words, $\mathbf{v}\in\Gamma_{i}\cap f_{\mathbf{g}}^{-1}(n)$. Recall that $\mathbf{x}^{i}\in\operatorname*{argmin}_{\mathbf{x}\in\Gamma_{i}\cap f_{\mathbf{g}}^{-1}(n)}x_{i}$. Hence, $\mathbf{v}\in\Gamma_{i}\cap f_{\mathbf{g}}^{-1}(n)$ implies that $x_{i}^{i}\geq v_{i}$, and in turn, $u^{L}(\mathbf{x},n)\leq u^{L}(\mathbf{x}^{i},n)$ according to Figure 4. Since $\max_{\mathbf{x}\in f_{\mathbf{g}}^{-1}(n)}u^{L}(\mathbf{x},n)$ is always attained at some vertex of $f_{\mathbf{g}}^{-1}(n)$, the stated result then follows.

Pick an arbitrary vertex $\mathbf{v}$ of $f_{\mathbf{g}}^{-1}(n)$. Since $\mathbf{v}\in\mathbb{R}^{m}$, it lies in $m$ hyperplanes of $f_{\mathbf{g}}^{-1}(n)$ and is uniquely defined by them. By definition, $f_{\mathbf{g}}^{-1}(n)$ is defined by the following linear constraints, each corresponding to a hyperplane:

[TABLE]

Hence, without loss of generality, we can assume that the $m$ hyperplanes defining $\mathbf{v}$ corresponds to the following coefficient matrix $A$:

[TABLE]

where we let $\tilde{\mu}(i,j)=W-\mu(i,j)$ for all $i,j$. Namely, $A\cdot\mathbf{v}=\mathbf{b}$, where $\mathbf{b}$ is the vector of the constant terms in (28). In what follows, we let $A_{i:j}$ denote the submatrix formed by the $i$-th to the $j$-th rows of $A$; and let $\overline{A}_{i:j}$ denote the submatrix formed by the $i$-th to the $j$-th columns. Since $\mathbf{v}$ is uniquely defined by $A$, we have $\operatorname{rank}\left(A\right)=m$.

We next argue that $\ell_{1}\geq m_{1}-2$ to complete the proof. This indicates that for at least $m_{1}-2$ actions $i\in[m_{1}-1]$, we have that $v_{i}=0$. In other words, $v_{i}>0$ for at most one $i\in[m_{1}-1]$, so by definition this means that $\mathbf{v}\in\Gamma_{i}$ for some $i\in[m_{1}-1]$.

Suppose for the sake of contradiction that $\ell_{1}\leq m_{1}-3$. Hence, $\ell_{2}+k=m-\ell_{1}-1\geq m-m_{1}+2$. Let

[TABLE]

be a submatrix of $A$, and let $\mathbf{v}^{\prime}=(v_{m_{1}+\ell_{2}-1},\dots,v_{m})$. Hence, $U$ is a $k$-by-$k^{\prime}$ matrix, with $k^{\prime}\leq k-1$.

We have $U\cdot\mathbf{v}^{\prime}=\mathbf{1}\cdot\lambda$, where $\lambda=\sum_{i=1}^{m_{1}-1}g_{i}\cdot v_{i}$. Note that $\mathbf{v}^{\prime}$ is a size-$k^{\prime}$ vector, so it must be that $\operatorname{rank}\left(U\mid\mathbf{1}^{\top}\right)\leq k^{\prime}\leq k-1$ (otherwise, the system of linear equations $U\cdot\mathbf{x}=\mathbf{1}\cdot\lambda$ would have no solution). Note that via linear transformation, the submatrix $A_{m-k+1,m}$ can be transformed into $\begin{pmatrix}O&\mathbf{1}^{\top}&U\end{pmatrix}$, where $O$ denotes a $k$-by-$(m_{1}+\ell_{2}-2)$ matrix with all entries being [math]. Hence, we get that

[TABLE]

Consequently,

[TABLE]

which contradicts the fact that $\operatorname{rank}\left(A\right)=m$. ∎

{restatable}

lemmalmmIgempty

There exists $N>0$, such that $I_{\mathbf{g}}\neq\emptyset$ if $g_{i}\geq N$ for all $i\in[m_{1}-1]$. Moreover, $N$ can be computed in polynomial time.

Proof.

Given Section 4.1, it suffices to find a value $N$ and prove that action $n$ is an SSE response of game $(u^{L},\tilde{u}_{\mathbf{g}}^{F})$ if $g_{i}\geq N$ for all $i\in[m_{1}-1]$.

Let $D=\max_{i,j,i^{\prime},j^{\prime}}|u^{L}(i,j)-u^{L}(i^{\prime},j^{\prime})|$. For every $\mathbf{x}\in\Delta_{m}$, we have

[TABLE]

where we used the fact that $u^{L}(i,n)\geq M_{\{n\}}-D$ for all $i$.

Moreover, let

[TABLE]

where $h$ is the BR-correspondence of $\mu$. For every $j\in[n-1]$,

[TABLE]

Recall that since $\mu$ is a cover of $\{n\}$, by definition $T<M_{\{n\}}$. So if it holds that $f_{\mathbf{g}}(\mathbf{x})=\{n\}$ for every $\mathbf{x}$ such that

[TABLE]

we know that $n$ must be an SSE response of $(u^{L},\tilde{u}^{F}_{\mathbf{g}})$; indeed, for all $\mathbf{y}$ such that $\sum_{i=1}^{m_{1}-1}y_{i}\leq c$, this leads to

[TABLE]

for all $j\in[n-1]$. It suffices to let

[TABLE]

for every $i\in[m_{1}-1]$ to ensure this, so any $N\geq\overline{N}$ satisfies the condition in the statement of this lemma.

To compute $N$, we can start from an arbitrary value $N>0$ and $\mathbf{g}=(N,\dots,N)$. We query oracle $\mathcal{A}_{\textnormal{SSE}}$ to check if $n$ is an SSE response of $(u^{L},\tilde{u}_{\mathbf{g}}^{F})$. Set $N\leftarrow 2\cdot N$ and $\mathbf{g}\leftarrow(N,\dots,N)$, and repeat the step if it is not. This way, we can find a value $N\geq\overline{N}$ in time $\log(\overline{N})$. ∎

{restatable}

lemmalmmIgnonempty

Suppose that $I_{\mathbf{g}}\neq\emptyset$ and $i\in[m_{1}-1]\setminus I_{\mathbf{g}}$. There exists $g^{*}>0$, such that $I_{\mathbf{g}^{\prime}}=I_{\mathbf{g}}\cup\{i\}$, where $\mathbf{g}^{\prime}=(g_{1},\dots,g_{i-1},g^{*},g_{i+1},\dots,g_{m_{1}-1})$. Moreover, $g^{\prime}$ can be computed in polynomial time.

Proof sketch.

Let

[TABLE]

where $u^{*}$ and $\mathbf{y}^{*}$ are the optimal solution to the following LP, where $V\coloneqq\max_{j\in[n]}\max_{\mathbf{x}\in f_{\mathbf{g}}^{-1}(j)}u^{L}(\mathbf{x},j)$ is the leader’s SSE payoff in $(u^{L},\tilde{u}_{\mathbf{g}}^{F})$.

[TABLE]

We demonstrate the following results to complete the proof; the omitted details can be found in the appendix.

• •

$g^{*}$ is well-defined (Lemma B.1), i.e., LP (30) is feasible and bounded, and $y^{*}_{i}>0$.

• •

$g^{*}>g_{i}$, so that by setting $g_{i}$ to $g^{*}$ we indeed increases its value (Lemma B.3). This is important because it means that $g_{i}>0$ still holds after the value change, so that we can apply Figure 4 inductively.

• •

Finally, let $(g^{\prime},\mathbf{g}_{-i})=(g_{1},\dots,g_{i-1},g^{\prime},g_{i+1},\dots,g_{m_{1}-1})$. Then $g^{\prime}=g^{*}$ is the only point where $I_{\mathbf{g}^{\prime}}=I_{\mathbf{g}}\cup\{i\}$ (Lemma B.4). This means that we can use binary search to pin down $g^{*}$. In particular, since ${g^{*}}$ is derived immediately from the solution to LP (30), its bit-size is bounded by a polynomial in the size of the LP. We can find ${g^{*}}$ in polynomial time. Note that we cannot compute $g^{*}$ directly using the expression (29) because LP (30) involves unknown parameters. ∎

B.4 Proof of Section B.3: Omitted Results

Lemma B.1**.**

LP (30) is feasible and bounded, and $y^{*}_{i}>0$.

Proof.

Notice that (30) is equivalent to computing the minimal value of $\max_{j\in[n-1]}\tilde{u}^{F}_{\mathbf{g}}(\mathbf{y},j)$ with $\mathbf{y}$ satisfying (30a) and (30c). It is feasible and bounded if there exists $\mathbf{y}\in\Gamma_{i}$ such that $u^{L}(\mathbf{y},n)=V$. Note that for all $\mathbf{y}\in\Gamma_{i}$, we have

[TABLE]

Let $\mathbf{z}$ be an arbitrary point such that $\sum_{k=m_{1}}^{m}z_{k}=1$ (hence $\mathbf{z}\in\Gamma_{i}$).

• •

When $\mathbf{y}=\mathbf{z}$, we have $u^{L}(\mathbf{y},n)=M_{\{n\}}$.

• •

When $\mathbf{y}=\mathbf{x}^{i}$, we have $u^{L}(\mathbf{y},n)=u^{L}(\mathbf{x}^{i},n)<V$ as $(\mathbf{x}^{i},n)$ is not an SSE by assumption.

By definition (i.e., (12)), $\mathbf{x}^{i}\in\Gamma_{i}$. By assumption $I_{\mathbf{g}}\neq\emptyset$, so $(\mathbf{x}^{k},n)$ is an SSE response for some $k\in[m_{1}-1]$. Hence, by definition,

[TABLE]

where $u^{L}(\mathbf{x}^{k},n)<\max_{\mathbf{x}\in\Gamma_{k}}u^{L}(\mathbf{x},n)$ follows by the fact that $u^{L}(\mathbf{x},n)$ decreases with $x_{k}$ in the space $\Gamma_{k}$ (Figure 4), and $x_{k}^{k}>0$ (Figure 4). Therefore, by continuity, there must exist $\mathbf{y}\in\Gamma_{i}$ such that $u^{L}(\mathbf{y},n)=V$. Moreover, for all such $\mathbf{y}$, $y_{i}>0$ as otherwise $u^{L}(\mathbf{y},n)=M_{\{n\}}>V$. ∎

Lemma B.2**.**

$u^{L}(\mathbf{x}^{i},n)$* increases strictly with $g_{i}$.*

Proof.

Pick two arbitrary values $g^{\prime},g^{\prime\prime}\in\mathbb{R}$ such that $g^{\prime}<g^{\prime\prime}$. Let

[TABLE]

Let ${\mathbf{x}^{\prime}}^{k}$ and ${\mathbf{x}^{\prime\prime}}^{k}$ denote the critical points defined with respect to $\mathbf{g}^{\prime}$ and $\mathbf{g}^{\prime\prime}$, respectively (i.e., (12)). Suppose for the sake of contradiction that $u^{L}({\mathbf{x}^{\prime}}^{i},n)\geq u^{L}({\mathbf{x}^{\prime\prime}}^{i},n)$.

By Figure 4, $u^{L}({\mathbf{x}^{\prime}}^{i},n)\geq u^{L}({\mathbf{x}^{\prime\prime}}^{i},n)$ implies that ${x^{\prime}_{i}}^{i}\leq{x^{\prime\prime}_{i}}^{i}$. Moreover,

[TABLE]

where we used the fact that ${x_{i}^{\prime}}^{i}>0$ (Figure 4).

Since $n\in f_{\mathbf{g}^{\prime}}({\mathbf{x}^{\prime}}^{i})$, by definition $\tilde{u}_{\mathbf{g}^{\prime}}^{F}({\mathbf{x}^{\prime}}^{i},n)\geq\tilde{u}_{\mathbf{g}^{\prime}}^{F}({\mathbf{x}^{\prime}}^{i},j)$ for all $j\in[n-1]$. Hence, using (31), we have

[TABLE]

where the last equality holds since $\tilde{u}_{\mathbf{g}}^{F}(\cdot,j)$ does not depend on $\mathbf{g}$ by construction. This means that $\tilde{u}_{\mathbf{g}^{\prime\prime}}^{F}(\mathbf{x},n)>\tilde{u}_{\mathbf{g}^{\prime\prime}}^{F}(\mathbf{x},j)$ for $\mathbf{x}$ in a neighborhood $\mathcal{N}$ of ${\mathbf{x}^{\prime}}^{i}$. Since ${x_{i}^{\prime}}^{i}>0$, ${\mathbf{x}^{\prime}}^{i}$ must lie in the relative interior of $\Gamma_{i}$. So $\mathcal{N}\cap\Gamma_{i}$ must contain a point $\mathbf{x}$ such that $x_{i}<{x_{i}^{\prime}}^{i}$. We then establish the following contradictory transitions:

[TABLE]

This completes the proof. ∎

Hereafter, we let $\mathbf{g}^{*}=(g_{1},\dots,g_{i-1},g^{*},g_{i+1},g_{m_{1}-1})$, and let ${\mathbf{x}^{*}}^{k}$ be the critical points defined with respect to $\mathbf{g}^{*}$ (i.e., (12)).

Lemma B.3**.**

$u^{L}({\mathbf{x}^{*}}^{i},n)=V$, and thus ${g^{*}}>g_{i}$.

Proof.

According to Lemma B.2, since $u^{L}(\mathbf{x}^{i},n)$ increases strictly with $g_{i}$, if $u^{L}({\mathbf{x}^{*}}^{i},n)=V>u^{L}({\mathbf{x}}^{i},n)$, then $g^{*}>g_{i}$.

Since $\mathbf{y}^{*}\in\Gamma_{i}$ (by (30a)), we have

[TABLE]

Note that $\tilde{u}_{\mathbf{g}^{\prime}}^{F}(\cdot,j)$ is not dependent on $\mathbf{g}^{\prime}$ for all $j\in[n-1]$. Hence, since $\mathbf{y}^{*}$ and $u^{*}$ satisfy (30b), we have

[TABLE]

As a result, $\tilde{u}_{\mathbf{g}^{*}}^{F}(\mathbf{y}^{*},n)\geq\tilde{u}_{\mathbf{g}^{*}}^{F}(\mathbf{y}^{*},j)$ for all $j\in[n]$, which means that $\mathbf{y}^{*}\in f_{\mathbf{g}^{*}}^{-1}(n)$. Using Figure 4, we have $u^{L}({\mathbf{x}^{*}}^{i},n)\geq u^{L}(\mathbf{y}^{*},n)=V$, where $u^{L}(\mathbf{y}^{*},n)=V$ follows by (30c).

To see that $u^{L}({\mathbf{x}^{*}}^{i},n)\leq V$, suppose for the sake of contradiction that $u^{L}({\mathbf{x}^{*}}^{i},n)>V$. Since $u^{L}(\mathbf{y}^{*},n)=V$, using Lemma 4 again, we have that ${x_{i}^{*}}^{i}<y_{i}^{*}$. Consider a point

[TABLE]

where $\lambda\in[0,1]$, and $\mathbf{e}^{i}=(e_{1}^{i},\dots,e_{m}^{i})$ such that $e_{i}^{i}=1$ and $e_{k}^{i}=0$ for all $k\neq i$. Since both ${\mathbf{x}^{*}}^{i}$ and $\mathbf{e}^{i}$ are in $\Gamma_{i}$, we have $\mathbf{z}\in\Gamma_{i}$. Moreover, since $e_{i}^{i}=1\geq y_{i}^{*}$, there exists $\lambda\in[0,1)$ with which we have $z_{i}=y_{i}^{*}$, and in turn

[TABLE]

and

[TABLE]

Meanwhile, note that for all $j\in[n-1]$, we have

[TABLE]

In addition, since ${\mathbf{x}^{*}}^{i}\in f_{\mathbf{g}^{*}}^{-1}(n)$, by definition we have

[TABLE]

Consequently, since $\lambda<1$, we must have

[TABLE]

Plugging into (32) gives $\tilde{u}_{\mathbf{g}^{*}}^{F}(\mathbf{z},j)<u^{*}$ for all $j\in[n-1]$. Pick $u\in[\max_{j\in[n-1]}\tilde{u}_{\mathbf{g}^{*}}^{F}(\mathbf{z},j),u^{*})$. The fact that $\mathbf{z}\in\Gamma_{i}$ and $u^{L}(\mathbf{z},n)=V$ (i.e., (33)) we showed above implies that $u$ and $\mathbf{z}$ form a feasible solution to LP (30). The objective value of this solution, i.e., $u$, is strictly smaller than $u^{*}$, which is a contradiction. This completes the proof of Lemma B.3. ∎

Lemma B.4**.**

$I_{\mathbf{g}^{\prime}}=I_{\mathbf{g}}$* if $g^{\prime}\in[g_{i},{g^{*}})$, $I_{\mathbf{g}^{\prime}}=\{i\}$ if $g^{\prime}>{g^{*}}$, and $I_{\mathbf{g}^{\prime}}=I_{\mathbf{g}}\cup\{i\}$ if $g^{\prime}={g^{*}}$.*

Proof.

We first argue that the following results hold for any $g^{\prime}\geq g_{i}$.

• (i)

$\max_{\mathbf{x}\in f_{\mathbf{g}^{\prime}}^{-1}(j)}u^{L}(\mathbf{x},j)\leq\max_{\mathbf{x}\in f_{\mathbf{g}}^{-1}(j)}u^{L}(\mathbf{x},j)$, for all $j\in[n-1]$; and

• (ii)

$u^{L}({\mathbf{x}^{\prime}}^{k},n)=u^{L}(\mathbf{x}^{k},n)$, for all $k\in[m_{1}-1]\setminus\{i\}$.

In words, (i) says that the leader’s payoffs for inducing a best response $j\in[n-1]$ does not increase with $g_{i}$; (ii) says that the leader’s payoffs for $(\mathbf{x}^{k},n)$, $k\neq i$, does not change with $g_{i}$.

To see (i), note that by construction, $\tilde{u}_{\mathbf{g}}^{F}(\mathbf{x},j)$ does not depend on $g_{i}$ for all $j\in[n-1]$. Moreover, now that $g^{\prime}\geq g_{i}$, $\tilde{u}_{\mathbf{g}^{\prime}}^{F}(\mathbf{x},n)\geq\tilde{u}_{\mathbf{g}}^{F}(\mathbf{x},n)$ for all $\mathbf{x}\in\Delta_{m}$. Therefore, any leader strategy $\mathbf{x}$ that does not induce $j$ under $\tilde{u}_{\mathbf{g}}^{F}$ does not induce $j$ under $\tilde{u}_{\mathbf{g}^{\prime}}^{F}$, either. We have $f_{\mathbf{g}^{\prime}}^{-1}(j)\subseteq f_{\mathbf{g}}^{-1}(j)$, so (i) follows immediately.

To see (ii), note that $\tilde{u}_{\mathbf{g}}^{F}$ does not depend on $g_{i}$ in the space $\Gamma_{k}$, $k\neq i$. Hence, $\Gamma_{k}\cap f_{\mathbf{g}^{\prime}}^{-1}(n)=\Gamma_{k}\cap f_{\mathbf{g}}^{-1}(n)$. By Lemma 4, the statement then follows.

We proceed with the proof. By assumption, $I_{\mathbf{g}}\neq\emptyset$ and we can assume that $1\in I_{\mathbf{g}}$. Hence, $u^{L}(\mathbf{x}^{1},n)=V$. The statements above then imply that

[TABLE]

where $\max_{j\in[n-1]}\max_{\mathbf{x}\in f_{\mathbf{g}}^{-1}(j)}u^{L}(\mathbf{x},j)\leq V$ follows by the definition of the SSE, and $u^{L}({\mathbf{x}^{\prime}}^{1},n)\leq\max_{\mathbf{x}\in f_{\mathbf{g}^{\prime}}^{-1}(n)}u^{L}(\mathbf{x},n)$ since $\mathbf{x}^{\prime-1}\in f_{\mathbf{g}^{\prime}}^{-1}(n)$ by definition. This further means that

[TABLE]

and applying Section 4.1, we get that

[TABLE]

Consequently,

• •

when $g^{\prime}\in[g_{i},g^{*})$, we have $\max_{k\in[m_{1}-1]}u^{L}({\mathbf{x}^{\prime}}^{k},n)=V>u^{L}({\mathbf{x}^{\prime}}^{i},n)$;

• •

when $g^{\prime}=g^{*}$, we have $\max_{k\in[m_{1}-1]}u^{L}({\mathbf{x}^{\prime}}^{k},n)=V=u^{L}({\mathbf{x}^{\prime}}^{i},n)$; and

• •

when $g^{\prime}>g^{*}$, we have $\max_{k\in[m_{1}-1]}u^{L}({\mathbf{x}^{\prime}}^{k},n)=u^{L}({\mathbf{x}^{\prime}}^{i},n)>V$.

Lemma B.4 then follows. ∎

B.5 Proof of Theorem 4.2

\lmmfindcovermu

Proof.

To decide whether $\mu$ is a cover of $S$, we check the satisfiability of the following linear constraints (by assumption of Theorem 4.2, $\mathcal{M}_{S}$ is given as a set of linear constraints):

[TABLE]

which can be done in polynomial time. Specifically:

• •

If the constraints are satisfiable, then there exists $\mathbf{y}\in\mathcal{M}_{S}$ such that $\mu(\mathbf{y},j)\leq 0=\mu(\mathbf{y},k)$ for all $j\in[n],k\in S$; hence, $k\in h(\mathbf{y})$, where $h$ denotes the BR-correspondence of $\mu$. We have

[TABLE]

By definition, this means that $\mu$ is not a cover of $S$.

• •

Conversely, suppose that the constraints are not satisfiable. Pick arbitrary $\mathbf{x}\in\mathcal{M}_{S}$ and $j\in h(\mathbf{x})$. Hence, $\mu(\mathbf{x},j)>0$, and according to the definition of $\mu$, it must be that $j\notin S$. We have $\mu(\mathbf{x},j)=\mathbf{b}_{j}\cdot\mathbf{x}-c_{j}>0$, which implies that $u^{L}(\mathbf{x},j)<M_{S}$ by (14). Since the choice of $\mathbf{x}$ and $k$ is arbitrary, we then have

[TABLE]

so $\mu$ is a cover of $S$.

Next, consider the second part of the statement. Suppose that $S$ admits a cover. We show that $\mu$ must be a cover of it. According to the necessary condition demonstrate in Definition 4, we have $M_{S}>M_{[n]}$. Hence, for any $\mathbf{x}\in\mathcal{M}_{S}$, $\min_{j\in[n]}u^{L}(\mathbf{x},j)\leq M_{[n]}<M_{S}$, which means that there exists $j\in[n]\setminus S$ such that $u^{L}(\mathbf{x},j)<M_{S}$. By (14), we then have $\mathbf{b}_{j}\cdot\mathbf{x}>c_{j}$, which means

[TABLE]

Namely, the linear constraints we presented above are not satisfiable. Therefore, $\mu$ is a cover of $S$. ∎

B.6 Proof of Theorem 4.2

\lmmfindcoverbbjcj

Proof.

Without loss of generality, we show how to learn $\mathbf{b}_{n}$ and $c_{n}$, and we assume that $n\notin S$. We first define the following parameters:

[TABLE]

Then specify $\mathbf{b}_{n}$ and $c_{n}$ as follows:

• •

If $n\notin Q$, we let $\mathbf{b}_{n}=\mathbf{a}_{n}$ and $c_{n}=\begin{cases}\check{d}-1,&\text{ if }d^{*}<\check{d};\\ d^{*},&\text{ if }d^{*}\in[\check{d},\hat{d}];\\ \hat{d},&\text{ if }d^{*}>\hat{d}.\end{cases}$. (By assumption of Theorem 4.2, $\mathbf{a}_{n}$ is given if $n\notin S\cup Q$.)

• •

If $n\in Q$, we let $b_{n,i}=\begin{cases}1,&\text{if }u^{L}(i,n)=M_{\{n\}}\\ 0,&\text{otherwise}\end{cases}$, and $c_{n}=\check{d}$.

Note that $\check{d}$ and $\hat{d}$ can be computed directly given that $\mathcal{M}_{S}$ is known (by assumption of Theorem 4.2). Nevertheless, $d^{*}$ cannot be computed directly since $\beta_{n}$, $\gamma_{n}$, and $M_{S}$ are unknown. To complete the proof, we show how to learn $d^{*}$ next.

Define the following payoff function parameterized by a number $d$. {boxedtext} For every $\mathbf{x}\in\Delta_{m}$,

[TABLE]

where we use arbitrary $k\in\operatorname*{argmax}_{\ell\in[n-1]}\tilde{u}^{F}(\mathbf{z},\ell)$, defined with arbitrarily selected

[TABLE]

$\mathbf{z}$ is well-defined if $d\in[\check{d},\hat{d}]$.

We argue that $n$ is an SSE response of the game $(u^{L},\tilde{u}_{d}^{F})$ if and only if $d\geq d^{*}$, so that we can use binary search to find out $d^{*}$ (or find out if $d^{*}<\check{d}$ or $d^{*}\geq\hat{d}$, in which case we only need $\check{d}$ and $\hat{d}$ to compute $\mathbf{b}_{n}$ and $c_{n}$ as defined above).

Denote the BR-correspondences of $\tilde{u}^{F}_{d}$ and $\tilde{u}^{F}$ as $f_{d}$ and $f$, respectively. Note the following facts:

1. (a)

$f_{d}(\mathbf{z})=f(\mathbf{z})\cup\{n\}$. 2. (b)

$f_{d}^{-1}(j)\subseteq f^{-1}(j)$ for all $j\in[n-1]$. 3. (c)

$\max_{\mathbf{x}\in f_{d}^{-1}(n)}\mathbf{b}_{n}\cdot\mathbf{x}=d$ if $d\in[\check{d},\hat{d}]$. 4. (d)

$u^{L}(\mathbf{x},j)=M_{S}$ for all $\mathbf{x}\in\mathcal{M}_{S}$ and $j\in f(\mathbf{x})$.

Indeed, (a) can be verified by comparing $\tilde{u}_{d}^{F}(\mathbf{z},j)$, $j\in[n-1]$, with $\tilde{u}^{F}(\mathbf{z},n)$. (b) is due to the fact that $f^{-1}(n)=\emptyset$ according to the property of $\tilde{u}^{F}$ as a base function.

To see (c), note that $n\in f_{d}(\mathbf{z})$ implies that

[TABLE]

Moreover, for all $\mathbf{x}\in f_{d}^{-1}(n)$,

[TABLE]

which implies $\mathbf{b}_{n}\cdot\mathbf{x}\leq d$ according to (34). Hence, $\max_{\mathbf{x}\in f_{d}^{-1}}\mathbf{b}_{n}\cdot\mathbf{x}=d$.

Finally, if (d) did not hold, then $u^{L}(\mathbf{x},j)\neq M_{S}$ for some $\mathbf{x}\in\mathcal{M}_{S}$ and $j\in f(\mathbf{x})$. Since $\tilde{u}^{F}$ is a base function, by definition, $j\in f(\mathbf{x})\subseteq S$. Moreover, $\mathbf{x}\in\mathcal{M}_{S}$ means that $\min_{k\in S}u^{L}(\mathbf{x},k)=M_{S}$, so we must have $u^{L}(\mathbf{x},j)>M_{S}$, which contradicts the definition of a base function.

For each $j\in[n]$, define

[TABLE]

which is the leader’s maximum attainable payoff for inducing a follower response $j$. According to (b), for all $j\in[n-1]$, we have

[TABLE]

where the second transition follows by the property of $\tilde{u}^{F}$ as a base function. Moreover, pick arbitrary $j\in f(\mathbf{z})$; by (a), (d), and the fact that $\mathbf{z}\in\mathcal{M}_{S}$, we get that

[TABLE]

Therefore, $\max_{j^{\prime}\in[n-1]}V_{d}^{L}(j^{\prime})=M_{S}$. By (c), we also have $\coloneqq V_{d}^{L}(n)=\gamma_{n}\cdot d+\beta_{n}$ if $d\in[\check{d},\hat{d}]$. It follows that

[TABLE]

is continuous and strictly increasing with respect to $d$. We can then use binary search and oracle $\mathcal{A}_{\textnormal{SSE}}$ to pin down $d^{*}$ (or decide if $d^{*}<\check{d}$ or $d^{*}>\hat{d}$): $n$ is an SSE response of $(u^{L},\tilde{u}_{d}^{F})$ if and only if $d\geq d^{*}$; meanwhile, there exists an SSE response $j\in[n-1]$ if and only if $d\leq d^{*}$. ∎

Appendix C Omitted Proofs in Section 5

C.1 Proof of Definition 6

\lmmtildeuLJ

Proof.

Note that for all $j\in[n]$, we have

[TABLE]

Specifically, (35) follows by the definition of $\widehat{J}$: if $j\notin\widehat{J}$, then by definition we have $M_{\{1\}}\leq\min_{\mathbf{x}\in\Delta_{m}}u^{L}(\mathbf{x},j)$; hence,

[TABLE]

(where $M_{[n]}<M_{\{1\}}$ is due to Item 5.1(a)). Moreover, if $j\notin\widehat{J}$, according to (15), we have

[TABLE]

Hence, (36) holds.

For all $j\in\widehat{J}$, we have

[TABLE]

which means that the following two statements are equivalent:

[TABLE]

That is, for all $j\in\widehat{J}$, (37) $\Longleftrightarrow$ (38). Using this equivalence and (35) and (36), we establish the following transitions to complete the proof:

[TABLE]

Appendix D Omitted Proofs in Section 5.1

D.1 An Useful Observation Used in All proofs

In all the proofs in Section 5.1, we make the following key assumption to ease our presentation.

Lemma D.1**.**

Without loss of generality, we can assume that there exists $i\in I_{2}$ such that $u^{L}(i,2)>u^{L}(i,1)$. (Recall that $I_{j}\coloneqq\operatorname*{argmax}_{i^{\prime}\in[m]}u^{L}(i^{\prime},j)$ for all $j\in[n]$.)

Proof.

To see the rationale of this assumption, consider the case where it does not hold, i.e., $u^{L}(i,1)\geq u^{L}(i,2)$ for all $i\in I_{2}$. By Item 5.1(b), $M_{\{2\}}\geq M_{\{1\}}$; hence, we have

[TABLE]

where $u^{L}(i,2)=M_{\{2\}}$ since $i\in I_{2}$. This means $M_{\{1\}}=M_{\{2\}}$ and $u^{L}(i,1)=u^{L}(i,2)$ for all $i\in I_{2}$. So $I_{2}\subseteq I_{1}$.

Hence, if there exists $i\in I_{1}$ such that $u^{L}(i,1)>u^{L}(i,2)$, we can exchange the roles of actions $1$ and $2$ so that all the assumptions made will hold (in particular, $M_{\{1\}}=M_{\{2\}}$ means that Item 5.1(b) still holds as $2\in\operatorname*{argmin}_{j\in[m]}M_{\{j\}}$). We can then proceed with the subsequent algorithm and eventually learn $\gamma_{1}/\gamma_{2}$ and $(\beta_{1}-\beta_{2})/\gamma_{2}$—from which the original target quantities $\gamma_{2}/\gamma_{1}$ and $(\beta_{2}-\beta_{1})/\gamma_{1}$ can be derived readily.

If otherwise $u^{L}(i,1)\leq u^{L}(i,2)$ for all $i\in I_{1}$, then we have $M_{\{1\}}=u^{L}(i,1)\leq u^{L}(i,2)\leq M_{\{2\}}$, so it must be that $u^{L}(i,1)=u^{L}(i,2)$ for all $i\in I_{1}$. Given that $I_{2}\subseteq I_{1}$ as we argued, we have $I_{2}=I_{1}$ in this case. We further consider the following possibilities:

• •

$I_{j}=I_{1}$ for all $j\in\widehat{J}$. Pick arbitrary $i\in I_{1}$, we then have $u^{L}(i,j)=M_{\{j\}}\geq M_{\{1\}}$ for all $j\in\widehat{J}$; moreover, according to the definition of $\widehat{J}$, $u^{L}(i,j)\geq M_{\{1\}}$ for all $j\notin\widehat{J}$. We get that $\min_{j\in[n]}u^{L}(i,j)\geq M_{\{1\}}$ . It follows that $M_{[n]}=\max_{\mathbf{x}\in\Delta_{m}}\min_{j\in[n]}u^{L}(\mathbf{x},j)\geq\min_{j\in[n]}u^{L}(i,j)\geq M_{\{1\}}$, which contradicts Item 5.1(a).

• •

$I_{j}\neq I_{1}$ for some $j\in\widehat{J}$ (and hence, $I_{j}\neq I_{2}$). This means that we are able to learn $\gamma_{j}/\gamma_{i}$ and $(\beta_{j}-\beta_{i})/\gamma_{i}$ for both $i\in{\{1,2\}}$ with our subsequent algorithm (i.e., by putting $j$ in place of action $2$ and $i$ in place of action $1$). We can then derive $\gamma_{2}/\gamma_{1}$ and $(\beta_{2}-\beta_{1})/\gamma_{1}$ as follows:

[TABLE]

D.2 Proof of Figure 6

\lmmfdmain

To prove Figure 6, we define

[TABLE]

for each $j\in[n]$, which is the maximum payoff the leader can obtain by inducing the follower to respond with $j$. We also let $V_{d}^{L}(j)=-\infty$ if $f_{d}^{-1}(j)=\emptyset$. By construction $f_{d}^{-1}(j)=\emptyset$ for all $j\in[n]\setminus{\{1,2\}}$, so only actions $1$ and $2$ can be SSE responses. We prove that $V_{d}^{L}(1)\geq V_{d}^{L}(2)$ if and only if $d\leq d^{*}$ and $V_{d}^{L}(1)\leq V_{d}^{L}(2)$ if and only if $d\geq d^{*}$. Indeed, in the case where $d=d^{*}$ the following lemma shows that $V_{d^{*}}^{L}(1)=V_{d^{*}}^{L}(2)=M_{\{1,2\}}$.

Lemma D.2**.**

$V_{d^{*}}^{L}(1)=V_{d^{*}}^{L}(2)=M_{\{1,2\}}$.

We defer the proof of Lemma D.2 to Section D.3 and proceed with the proof of Figure 6.

Proof of Figure 6.

Clearly, $V_{d}^{L}(1)$ is non-increasing with respect to $d$. Moreover, we have $V_{d^{*}}^{L}(1)=V_{d^{*}}^{L}(2)$ according to Lemma D.2. Hence, it suffices to prove that $V_{d}^{L}(1)>V_{d}^{L}(2)$ for all $d<d^{*}$, and $V_{d}^{L}(1)<V_{d}^{L}(2)$ for all $d>d^{*}$.

If $d<d^{*}$, then by construction we have $\mathbf{a}_{2}\cdot\mathbf{x}\leq d<d^{*}$ for all $\mathbf{x}\in f_{d}^{-1}(2)$. Hence, $\max_{\mathbf{x}\in f_{d}^{-1}(2)}\mathbf{a}_{2}\cdot\mathbf{x}<d^{*}$, and

[TABLE]

where $V_{d^{*}}^{L}(1)\leq V_{d}^{L}(1)$ as $V_{d}^{L}(1)$ is non-increasing with respect to $d$.

If $d>d^{*}$, consider the following two cases.

• •

$f_{d}^{-1}(1)=\emptyset$. Then it follows immediately that $V_{d}^{F}(2)\geq V_{d^{*}}^{F}(2)=M_{\{1,2\}}>-\infty=V_{d}^{F}(1)$.

• •

$f_{d}^{-1}(1)\neq\emptyset$. Pick arbitrary $\mathbf{x}\in f_{d}^{-1}(1)$. By definition, we have $\mathbf{a}_{2}\cdot\mathbf{x}\geq d$. Since $V_{d^{*}}^{L}(2)=M_{\{1,2\}}$, there exists $\mathbf{y}\in\Delta_{m}$ such that $u^{L}(\mathbf{y},2)=M_{\{1,2\}}$, which means that $\mathbf{a}_{2}\cdot\mathbf{y}=(M_{\{1,2\}}-\beta_{2})/\gamma_{2}=d^{*}<d$. Hence, there exists a convex combination $\mathbf{z}$ of $\mathbf{x}$ and $\mathbf{y}$ such that $\mathbf{a}_{2}\cdot\mathbf{z}=d$. We have $\mathbf{z}\in f_{d}^{-1}(2)$, which implies that

[TABLE]

This completes the proof. ∎

D.3 Proof of Lemma D.2

We prove the lemma in two parts: (1) $V_{d^{*}}^{L}(1)=M_{\{1,2\}}$ (Lemma D.3), and (2) $V_{d^{*}}^{L}(2)=M_{\{1,2\}}$ (Lemma D.4). En route, we also prove a result (stated in Lemma D.4) that will be useful in the next section.

Lemma D.3**.**

$V_{d^{*}}^{L}(1)=M_{\{1,2\}}$.

Proof.

Suppose for the sake of contradiction that $V_{d^{*}}^{L}(1)\neq M_{\{1,2\}}$ and consider the following cases.

Case 1.

$V_{d^{*}}^{L}(1)<M_{\{1,2\}}$. Since $V_{d^{*}}^{L}(1)<M_{\{1,2\}}$, then for all $\mathbf{x}\in f_{d^{*}}^{-1}(1)$ we have

[TABLE]

For all $\mathbf{x}\in\Delta_{m}\setminus f_{d^{*}}^{-1}(1)$, by construction we have $\mathbf{a}_{2}\cdot\mathbf{x}<d^{*}$; hence,

[TABLE]

Therefore, for all $\mathbf{x}\in\Delta_{m}$, we have $\min_{j\in{\{1,2\}}}u^{L}(\mathbf{x},j)<M_{\{1,2\}}$. This leads to a contradiction: $M_{\{1,2\}}=\max_{\mathbf{x}\in\Delta_{m}}\min_{j\in{\{1,2\}}}u^{L}(\mathbf{x},j)<M_{\{1,2\}}$.

Case 2.

$V_{d^{*}}^{L}(1)>M_{\{1,2\}}$. We further consider the following two cases.

• (i)

There exists $\mathbf{y}_{2}\in f_{d^{*}}^{-1}(1)$ such that

[TABLE]

Pick arbitrary $\mathbf{y}_{1}\in\operatorname*{argmax}_{\mathbf{x}\in f_{d^{*}}^{-1}(1)}u^{L}(\mathbf{x},1)$. By assumption, $V_{d^{*}}^{L}(1)>M_{\{1,2\}}$, so

[TABLE]

The fact that $\mathbf{y}_{1}\in f_{d^{*}}^{-1}(1)$ also implies that $\mathbf{a}_{2}\cdot\mathbf{y}_{1}\geq d^{*}$ by construction. Hence,

[TABLE]

Let $\mathbf{z}=\lambda\cdot\mathbf{y}_{1}+(1-\lambda)\cdot\mathbf{y}_{2}$, where $\lambda\in(0,1)$. By continuity, (42) implies that when $\lambda$ is sufficiently close to $1$, we have $u^{L}(\mathbf{z},1)>M_{\{1,2\}}$. Moreover, (41), (43) and the fact that $\lambda<1$ imply that $u^{L}(\mathbf{z},2)>M_{\{1,2\}}$. Thus, $\min_{j\in{\{1,2\}}}u^{L}(\mathbf{z},j)>M_{\{1,2\}}$, which contradicts the definition of $M_{\{1,2\}}$, i.e., $M_{\{1,2\}}=\max_{\mathbf{x}\in\Delta_{m}}\min_{j\in{\{1,2\}}}u^{L}(\mathbf{x},j)\geq\min_{j\in{\{1,2\}}}u^{L}(\mathbf{z},j)$.

• (ii)

$u^{L}(\mathbf{x},2)\leq M_{\{1,2\}}$ for all $\mathbf{x}\in f_{d^{*}}^{-1}(1)$. Note that for all $\mathbf{x}\in\Delta_{m}\setminus f_{d^{*}}^{-1}(1)$, it holds that $u^{L}(\mathbf{x},2)<M_{\{1,2\}}$ (see (40)). Hence, now we have $u^{L}(\mathbf{x},2)\leq M_{\{1,2\}}$, for all $\mathbf{x}\in\Delta_{m}$, which implies that

[TABLE]

According to the assumption of Case 2, we have $V_{d^{*}}^{L}(1)>M_{\{1,2\}}$. It follows that

[TABLE]

This contradicts Item 5.1(b).

Therefore, both cases lead to contradictions. We have $V_{d^{*}}^{L}(1)=M_{\{1,2\}}$. ∎

Lemma D.4**.**

$V_{d^{*}}^{L}(2)=M_{\{1,2\}}$. Moreover, there exists $\mathbf{x}^{-}\in\Delta_{m}$ such that $\mathbf{a}_{2}\cdot\mathbf{x}^{-}<d^{*}$.

Proof.

According to Lemma D.3, $f_{d^{*}}^{-1}(1)\neq\emptyset$, which means that there exists $\mathbf{x}^{+}\in\Delta_{m}$ such that $\mathbf{a}_{2}\cdot\mathbf{x}^{+}\geq d^{*}$.

We next show that there exists $\mathbf{x}^{-}\in\Delta_{m}$ such that $\mathbf{a}_{2}\cdot\mathbf{x}^{-}<d^{*}$. Suppose for the sake of contradiction that $\mathbf{a}_{2}\cdot\mathbf{x}\geq d^{*}$ for all $\mathbf{x}\in\Delta_{m}$. By definition we then have $f_{d^{*}}^{-1}(1)=\Delta_{m}$, so we get that

[TABLE]

By Lemma D.3, $V_{d^{*}}^{L}(1)=M_{\{1,2\}}$, so we have

[TABLE]

Now that $\mathbf{a}_{2}\cdot\mathbf{x}\geq d^{*}$ for all $\mathbf{x}\in\Delta_{m}$ by assumption, we have

[TABLE]

Hence, $\min_{\mathbf{x}\in\Delta_{m}}u^{L}(\mathbf{x},2)\geq M_{\{1\}}$, which contradicts 5.2 and the definition of $\widehat{J}$.

Therefore, we obtain two points $\mathbf{x}^{+},\mathbf{x}^{-}\in\Delta_{m}$ such that $\mathbf{a}_{2}\cdot\mathbf{x}^{+}\geq d^{*}$ and $\mathbf{a}_{2}\cdot\mathbf{x}^{-}<d^{*}$. There must be a convex combination $\mathbf{y}$ of $\mathbf{x}^{-}$ and $\mathbf{x}^{+}$ such that $\mathbf{y}\in\Delta_{m}$ and $\mathbf{a}_{2}\cdot\mathbf{y}=d^{*}$. Hence, $\mathbf{y}\in f_{d^{*}}^{-1}(2)$ and

[TABLE]

where $d^{*}\geq\max_{\mathbf{x}\in f_{d^{*}}^{-1}(2)}\mathbf{a}_{2}\cdot\mathbf{x}$ holds as $f_{d^{*}}^{-1}(2)=\{\mathbf{x}\in\Delta_{m}:\mathbf{a}_{2}\cdot\mathbf{x}\leq d^{*}\}$ by construction. This means that $\max_{\mathbf{x}\in f_{d^{*}}^{-1}(2)}\mathbf{a}_{2}\cdot\mathbf{x}=d^{*}$. Consequently, $V_{d^{*}}^{L}(2)=\gamma_{2}\cdot d^{*}+\beta_{2}=M_{\{1,2\}}$. ∎

D.4 Proof of Figure 6

\thmcomputedstar

Proof.

Figure 6 implies immediately that we can use binary search and oracle $\mathcal{A}_{\textnormal{ER}}$ to compute $d^{*}$ in polynomial time. Moreover, both actions $1$ and $2$ are SSE responses of $\mathcal{G}_{d^{*}}=(u^{L},\tilde{u}_{d^{*}}^{F})$. Pick arbitrary $\mathbf{x}\in\operatorname*{argmax}_{\mathbf{x}^{\prime}\in f_{d^{*}}^{-1}(1)}u^{L}(\mathbf{x}^{\prime},1)$ and $\mathbf{y}\in\operatorname*{argmax}_{\mathbf{y}^{\prime}\in f_{d^{*}}^{-1}(2)}u^{L}(\mathbf{y}^{\prime},2)$. Then $(\mathbf{x},1)$ and $(\mathbf{y},2)$ are both SSEs; hence, $u^{L}(\mathbf{x},1)=u^{L}(\mathbf{y},2)$. Moreover, since $u^{L}(\mathbf{x},1)=V_{d^{*}}^{L}(1)$ and $u^{L}(\mathbf{y},2)=V_{d^{*}}^{L}(2)$, according to Lemma D.2, we have $u^{L}(\mathbf{x},1)=u^{L}(\mathbf{y},2)=M_{\{1,2\}}$.

Indeed, since $u^{L}(\mathbf{x},j)=\gamma_{j}\cdot\mathbf{x}+\beta_{j}$, to compute $\mathbf{x}$ and $\mathbf{y}$ amounts to solving $\max_{\mathbf{x}^{\prime}\in f_{d^{*}}^{-1}(1)}\mathbf{a}_{1}\cdot\mathbf{x}^{\prime}$ and $\max_{\mathbf{y}^{\prime}\in f_{d^{*}}^{-1}(2)}\mathbf{a}_{2}\cdot\mathbf{y}^{\prime}$, respectively. Moreover, by definition, the constraints $\mathbf{x}^{\prime}\in f_{d^{*}}^{-1}(1)$ and $\mathbf{y}^{\prime}\in f_{d^{*}}^{-1}(2)$ are further equivalent to $\mathbf{a}_{2}\cdot\mathbf{x}^{\prime}\geq d^{*}$ and $\mathbf{a}_{2}\cdot\mathbf{x}^{\prime}\leq d^{*}$, respectively. Hence, the task reduces to solving two LPs, which can be done in polynomial time. This completes the proof. ∎

Appendix E Omitted Proofs in Section 5.2

E.1 Results When ${\{1,2\}}$ does not Admit a Cover

Lemma E.1**.**

If ${\{1,2\}}$ does not admit a cover, then $J\subseteq[n]$ and $\mathbf{x}^{*}\in\Delta_{m}$ that satisfy (7) can be computed in polynomial time.

Proof.

By Definition 4, now that ${\{1,2\}}$ does not admit a cover, it must be that $M_{\{1,2\}}=M_{[n]}$. Let $J={\{1,2\}}$.

Next, pick arbitrary

[TABLE]

where $f_{d^{*}}$ is the BR-correspondence of (18). By definition, we have $u^{L}(\mathbf{z},1)=M_{\{1\}}$, and according to Lemma D.2, $u^{L}(\mathbf{y},1)=M_{\{1,2\}}$. Now that $M_{\{1,2\}}=M_{[n]}$, by Item 5.1(a), $M_{\{1\}}>M_{\{1,2\}}$. This further implies that $\mathbf{z}\notin f_{d^{*}}^{-1}(1)$. Hence, we have $\mathbf{a}_{2}\cdot\mathbf{y}\leq d^{*}=d_{2}^{*}$ and $\mathbf{a}_{2}\cdot\mathbf{z}\leq d^{*}=d_{2}^{*}$, which implies that there exists $\mathbf{x}^{*}$ in the line segment between $\mathbf{y}$ and $\mathbf{z}$ such that $\mathbf{a}_{2}\cdot\mathbf{x}^{*}=d_{2}^{*}$, or equivalently

[TABLE]

Since $u^{L}(\mathbf{x},1)\geq M_{\{1,2\}}$ for $\mathbf{x}\in\{\mathbf{y},\mathbf{z}\}$, by linearity of $u^{L}(\cdot,1)$, we also get that $u^{L}(\mathbf{x}^{*},1)\geq M_{\{1,2\}}$. At the same time, since $\mathbf{x}^{*}\in f_{d^{*}}^{-1}(1)$, it must be that $u^{L}(\mathbf{x}^{*},1)\leq u^{L}(\mathbf{y},1)=M_{\{1,2\}}$, according to the definition of $\mathbf{y}$. As a result,

[TABLE]

Hence, $J$ and $\mathbf{x}^{*}$ satisfy (7). To compute $\mathbf{x}^{*}$ amounts to finding an $\mathbf{x}\in\Delta_{m}$ such that $u^{L}(\mathbf{x},1)=u^{L}(\mathbf{x},2)=M_{\{1,2\}}$, which translates to the linear constraints $\mathbf{a}_{1}\cdot\mathbf{x}=d_{1}^{*}$ and $\mathbf{a}_{2}\cdot\mathbf{x}=d_{2}^{*}$. Hence, $\mathbf{x}^{*}$ can be computed in polynomial time. ∎

E.2 Proof of Section 5.2.1

First, the following lemma shows that Condition (ii) is equivalent to that $P_{1}(d_{1},d_{2})\neq\emptyset$ and $P_{2}(d_{1},d_{2})\neq\emptyset$.

{restatable}

lemmalmmPjnonempty

Suppose that $d_{1}\leq d_{1}^{*}$, and $d_{2}\leq d_{2}^{*}$. Then $V_{d_{1},d_{2}}^{L}(j)=\gamma_{j}\cdot d_{j}+\beta_{j}$ if $P_{j}(d_{1},d_{2})\neq\emptyset$ for $j\in{\{1,2\}}$.

Proof.

We prove the equivalent statement: $\max_{\mathbf{x}\in P_{j}(d_{1},d_{2})}\mathbf{a}_{j}\cdot\mathbf{x}=d_{j}$ if $P_{j}(d_{1},d_{2})\neq\emptyset$. Consider the case where $j=1$. Indeed, by definition we have $\mathbf{a}_{1}\cdot\mathbf{x}\leq d_{1}$ for all $\mathbf{x}\in P_{1}(d_{1},d_{2})$, so it suffices to show that $\max_{\mathbf{x}\in P_{1}(d_{1},d_{2})}\mathbf{a}_{1}\cdot\mathbf{x}\geq d_{1}$.

Choose arbitrary $\mathbf{x}\in P_{1}(d_{1},d_{2})$. By definition, we have

[TABLE]

By Lemma D.3, there exists $\mathbf{y}\in\Delta_{m}$ such that

[TABLE]

Hence, there exists a convex combination $\mathbf{z}$ of $\mathbf{x}$ and $\mathbf{y}$, such that $\mathbf{a}_{1}\cdot\mathbf{z}=d_{1}$ and $\mathbf{a}_{2}\cdot\mathbf{z}\geq d_{2}$, which means $\mathbf{z}\in P_{1}(d_{1},d_{2})$ and hence, $\max_{\mathbf{x}\in P_{1}(d_{1},d_{2})}\mathbf{a}_{1}\cdot\mathbf{x}\geq\mathbf{a}_{1}\cdot\mathbf{z}=d_{1}$.

The case where $j=2$ can be proven analogously. ∎

Next, we identify a boundary value $\epsilon^{\prime}$ that makes $P_{j}(d_{1},d_{2})\neq\emptyset$ for all $d_{j}\in[d_{1}^{*}-\epsilon^{\prime},d_{j}^{*}]$. This value is characterized as the optimal solution to the following LP according to Section E.2:

[TABLE]

The LP characterization is important as it helps us bound the bit-size of $\hat{\epsilon}$.

{restatable}

lemmalmmepsilonpgt

Let $\epsilon^{\prime}$ be the optimal value of LP (44). Then $\epsilon^{\prime}>0$. Moreover, $P_{1}(d_{1},d_{2})\neq\emptyset$ and $P_{2}(d_{1},d_{2})\neq\emptyset$ for all $d_{1}\in[d_{1}^{*}-\epsilon^{\prime},d_{1}^{*}]$ and $d_{2}\in[d_{2}^{*}-\epsilon^{\prime},d_{2}^{*}]$.

Proof.

Indeed, if $\epsilon^{\prime}>0$, then according to the constraints of LP (44), we have $P_{1}(d_{1}^{*}-\epsilon^{\prime},d_{2}^{*})\neq\emptyset$ and $P_{2}(d_{1}^{*},d_{2}^{*}-\epsilon^{\prime})\neq\emptyset$. The second part of the statement of this lemma then follows readily: for all $d_{1}\in[d_{1}^{*}-\epsilon^{\prime},d_{1}^{*}]$ and $d_{2}\in[d_{2}^{*}-\epsilon^{\prime},d_{2}^{*}]$, we have

[TABLE]

and

[TABLE]

Hence, it suffices to prove that $\epsilon^{\prime}>0$.

According to Lemma D.4, there exists $\mathbf{x}^{-}\in\Delta_{m}$ such that $\mathbf{a}_{2}\cdot\mathbf{x}^{-}<d^{*}=d_{2}^{*}$. Let $\epsilon_{2}^{\prime}=d_{2}^{*}-\mathbf{a}_{2}\cdot\mathbf{x}^{-}$. We have $\epsilon_{2}^{\prime}>0$. Moreover, $\mathbf{x}^{-}\in P_{2}(d_{1}^{*},d_{2}^{*}-\epsilon_{2}^{\prime})$, which means $P_{2}(d_{1}^{*},d_{2}^{*}-\epsilon_{2}^{\prime})\neq\emptyset$.

We next argue that there also exists $\epsilon_{1}^{\prime}>0$ such that $P_{1}(d_{1}^{*}-\epsilon_{1}^{\prime},d_{2}^{*})\neq\emptyset$. Once this holds, we have $\varepsilon=\min\{\epsilon_{1}^{\prime},\epsilon_{2}^{\prime}\}>0$. Moreover,

[TABLE]

and

[TABLE]

Hence, $\varepsilon>0$, along with two arbitrarily chosen points $\mathbf{x}_{1}$ and $\mathbf{x}_{2}$ from the above sets, constitutes a feasible solution to (44), which implies the claimed result.

We next demonstrate the existence of $\epsilon_{1}^{\prime}$. Note that it suffices to show that:

[TABLE]

Indeed, letting $\epsilon_{1}^{\prime}=d_{1}^{*}-\mathbf{a}_{1}\cdot\mathbf{x}^{*}$, we then have $\mathbf{x}^{*}\in P_{1}(d_{1}^{*}-\epsilon_{1}^{\prime},d_{2}^{*})$, which means $P_{1}(d_{1}^{*}-\epsilon_{1}^{\prime},d_{2}^{*})\neq\emptyset$. Consider the following two cases.

Case 1.

$\mathbf{a}_{2}\cdot\mathbf{x}\leq d_{2}^{*}$ for all $\mathbf{x}\in\Delta_{m}$. In this case, we can prove that

[TABLE]

by noting the following facts:

• •

For all $\mathbf{x}\in P_{1}(d_{1}^{*},d_{2}^{*})$, by definition we have $\mathbf{a}_{2}\cdot\mathbf{x}\geq d_{2}^{*}$. Now that $\mathbf{a}_{2}\cdot\mathbf{x}\leq d_{2}^{*}$, it must be that $\mathbf{a}_{2}\cdot\mathbf{x}=d_{2}^{*}$, which further means

[TABLE]

for all $\mathbf{x}\in P_{1}(d_{1}^{*},d_{2}^{*})$, where $M_{\{1,2\}}=V_{d_{1}^{*},d_{2}^{*}}^{L}(2)$ follows by Lemma D.4.

• •

For all $\mathbf{x}\notin P_{1}(d_{1}^{*},d_{2}^{*})$, we have $\mathbf{a}_{2}\cdot\mathbf{x}<d^{*}$, so $u^{L}(\mathbf{x},2)<M_{\{1,2\}}=V_{d_{1}^{*},d_{2}^{*}}^{L}(2)$.

Hence, by Lemma D.1, there exists $\mathbf{x}\in\operatorname*{argmax}_{\mathbf{x}^{\prime}\in\Delta_{m}}u^{L}(\mathbf{x}^{\prime},2)=P_{1}(d_{1}^{*},d_{2}^{*})$ such that $u^{L}(\mathbf{x},1)<u^{L}(\mathbf{x},2)$. Now that $\mathbf{a}_{2}\cdot\mathbf{x}\leq d^{*}$ for all $\mathbf{x}\in\Delta_{m}$ according to the assumption of Case 1, we have $u^{L}(\mathbf{x},2)\leq M_{\{1,2\}}$, and in turn $u^{L}(\mathbf{x},1)<M_{\{1,2\}}$. Therefore, letting $\mathbf{x}^{*}$ be the aforementioned $\mathbf{x}$ gives $\mathbf{a}_{1}\cdot\mathbf{x}^{*}<(M_{\{1,2\}}-\beta_{1})/\gamma_{1}=d_{1}^{*}$.

Case 2.

There exists $\mathbf{x}\in\Delta_{m}$ such that $\mathbf{a}_{2}\cdot\mathbf{x}>d_{2}^{*}$. Let $\hat{d}_{2}=\max_{\mathbf{x}\in\Delta_{m}}\mathbf{a}_{2}\cdot\mathbf{x}$; we have $\hat{d}_{2}>d_{2}^{*}$. Hence, let $\hat{d}_{2}^{\prime}=(\hat{d}_{2}+d_{2}^{*})/2$, we have $\hat{d}_{2}>\hat{d}_{2}^{\prime}>d_{2}^{*}$. By continuity, there exists $\mathbf{y}\in\Delta_{m}$ such that $\mathbf{a}_{2}\cdot\mathbf{y}=\hat{d}_{2}^{\prime}$.

Now consider a vector $\tilde{\mathbf{y}}\in\mathbb{R}^{m}$ such that

[TABLE]

where $\delta>0$ and $m^{\prime}=\left|i\in[m]:y_{i}=0\right|$. Clearly, $\sum_{i\in[m]}\tilde{y}_{i}=\sum_{i\in[m]}y_{i}=1$, and when $\delta$ is sufficiently close to [math], we can ensure that: $\tilde{y}_{i}\in(0,1)$ for all $i\in[m]$, so $\tilde{\mathbf{y}}$ is in the interior of $\Delta_{m}$; moreover, $\hat{d}_{2}>\mathbf{a}_{2}\cdot\tilde{\mathbf{y}}>d_{2}^{*}$ by continuity, so $\tilde{\mathbf{y}}\in P_{1}(d_{1}^{*},d_{2}^{*})$.

If it happens that $\mathbf{a}_{1}\cdot\tilde{\mathbf{y}}<d_{1}^{*}$, then we are done with $\tilde{\mathbf{y}}$ being a point satisfying (45). Hence, in what follows, we assume that $\mathbf{a}_{1}\cdot\tilde{\mathbf{y}}\geq d_{1}^{*}$. According to Lemma D.3, we have $\mathbf{a}_{1}\cdot\mathbf{x}\leq d_{1}^{*}$ for all $\mathbf{x}\in f_{d^{*}}^{-1}(1)=P_{1}(d_{1}^{*},d_{2}^{*})$ (where $f$ is the BR-correspondence of $\tilde{u}^{F}_{d}$ defined in (18)), so it must be that $\mathbf{a}_{1}\cdot\tilde{\mathbf{y}}=d_{1}^{*}$.

We proceed by defining a set of vectors $\mathbf{z}^{1},\dots,\mathbf{z}^{m}\in\mathbb{R}^{m}$ such that

[TABLE]

where $\delta^{\prime}>0$. When $\delta^{\prime}$ is sufficiently close to [math], we can ensure that $\hat{d}_{2}>\mathbf{a}_{2}\cdot\mathbf{z}^{i}>d_{2}^{*}$. Moreover, we have $\sum_{j\in[m]}z_{j}^{i}=\sum_{j\in[m]}\tilde{y}_{j}=1$, and since $\tilde{\mathbf{y}}$ is in the interior of $\Delta_{m}$, a sufficiently small $\delta^{\prime}$ also ensures that $\mathbf{z}^{i}\in\Delta_{m}$. Hence, $\mathbf{z}^{i}\in P_{1}(d_{1}^{*},d_{2}^{*})$. Observe that

[TABLE]

We claim that at least $\mathbf{a}_{1}\cdot\mathbf{z}^{i}<d_{1}^{*}$ for at least one $i\in[m]$. Indeed, if $\mathbf{a}_{1}\cdot\mathbf{z}^{i}\geq d_{1}^{*}$ for all $i\in[m]$, according to the same argument above via Lemma D.3, we get that $\mathbf{a}_{1}\cdot\mathbf{z}^{i}=d_{1}^{*}$ for all $i\in[m]$. It follows that $a_{1,1}=a_{1,2}=\dots=a_{1,m}=\sum_{j\in[m]}a_{1,j}/m$, which contradicts Item 5.1(c). ∎

Using the above results, we now prove Section 5.2.1.

Proof of Section 5.2.1.

We first solve LP (44) and let the optimal value be $\epsilon^{\prime}$. If some $j\in\{1,2\}$ is an SSE response of $\mathcal{G}_{d_{1}^{*}-{\epsilon^{\prime}},d_{2}^{*}-{\epsilon^{\prime}}}$, then we are done with $\hat{\epsilon}=\epsilon^{\prime}$ satisfying both conditions in Section 5.2.1. Specifically:

• •

Condition (ii) holds according to Section E.2 and Section E.2.

• •

Further applying Section E.2, we get that for any $d_{j}\in[d_{j}^{*}-\epsilon^{\prime},d_{j}^{*}]$,

[TABLE]

for both $j\in{\{1,2\}}$, whereas

[TABLE]

for all $k\in[n]\setminus{\{1,2\}}$ as $f_{d_{1},d_{2}}^{-1}(k)\subseteq f_{d_{1}^{*}-\hat{\epsilon},d_{2}^{*}-\hat{\epsilon}}^{-1}(k)$. In words, the leader’s payoff for playing a strategy that induced actions $1$ or $2$ will not decrease, while that for strategies inducing actions $k\notin{\{1,2\}}$ will not increase. Hence, at least one of $j\in{\{1,2\}}$ remains to be an SSE response, and Condition (i) also holds.

Hence, in what follows, we assume that neither action $1$ nor $2$ is an SSE response of $\mathcal{G}_{d_{1}^{*}-{\epsilon^{\prime}},d_{2}^{*}-{\epsilon^{\prime}}}$.

For each $k\in[n]\setminus\{1,2\}$, we define the following LP, where $\mu$ is the payoff function of $h$.

[TABLE]

Hence, we obtain a class of LPs. Let $\varepsilon$ be the minimum optimal value of LPs in this class that indeed has a feasible solution. We show that any $\hat{\epsilon}<\varepsilon$ satisfies the conditions in the statement of Section 5.2.1 via the following claims.

Claim 4**.**

If neither action $1$ nor $2$ is an SSE response of $\mathcal{G}_{d_{1}^{*}-{\epsilon^{\prime}},d_{2}^{*}-{\epsilon^{\prime}}}$, then LP (48) must be feasible for at least one $k\in[n]\setminus\{1,2\}$.

Proof.

Indeed, pick an arbitrary SSE $(\mathbf{x},k)$ of $\mathcal{G}_{d_{1}^{*}-{\epsilon^{\prime}},d_{2}^{*}-{\epsilon^{\prime}}}$. Since neither action $1$ nor $2$ is an SSE response, it must be that $k\in[n]\setminus{\{1,2\}}$. Consider the LP (48). $\epsilon^{\prime}$ and $\mathbf{x}$ form a feasible solution to this LP as all the constraints are satisfied: Constraints (48b) and (48c) holds because $k$ is a best response against $\mathbf{x}$; and Constraint (48d) says that $(\mathbf{x},k)$ gives the leader a higher payoff than the best payoff attainable by committing to any strategy in $P_{1}$ or $P_{2}$ (when $P_{1}$ and $P_{2}$ are non-empty, the right hand side of this constraint is the best attainable payoff in the corresponding region as argued in Section E.2). ∎

Claim 5**.**

$\varepsilon>0$.

Proof.

Suppose for the sake of contradiction that $\varepsilon\leq 0$.

If $\varepsilon<0$, then by Constraint (48b) there exists $\mathbf{x}\in\Delta_{m}$ such that $\mathbf{a}_{j}\cdot\mathbf{x}>d_{j}^{*}$ for $j\in{\{1,2\}}$ (i.e., $u^{L}(\mathbf{x},j)>M_{\{1,2\}}$), which would imply that $\max_{\mathbf{x}^{\prime}\in\Delta_{m}}\min_{j\in{\{1,2\}}}u^{L}(\mathbf{x}^{\prime},j)>M_{\{1,2\}}$—a contradiction.

If $\varepsilon=0$, then $P_{0}(d_{1}^{*}-\varepsilon,d_{2}^{*}-\varepsilon)=\mathcal{M}_{\{1,2\}}$, in which case all $\mathbf{x}$ satisfying Constraints (48b) and (48c) would violate (48d): Constraints (48b) and (48c) require that $\mathbf{x}\in P_{0}=\mathcal{M}_{\{1,2\}}$ and $k\in\widetilde{BR}(\mathbf{x})$, respectively, so

[TABLE]

where the second transition follows by the definition of a cover (see (9)). ∎

Claim 6**.**

Any $\hat{\epsilon}<\varepsilon$ satisfies the conditions in the statement of Section 5.2.1.

Proof.

According to 5, $\varepsilon>0$. Hence, $\hat{\epsilon}>0$. By Eq. 48a, we have $\hat{\epsilon}<\varepsilon\leq\epsilon^{\prime}$, so Condition (ii) holds according to Section E.2 and Section E.2.

By definition, $\varepsilon$ is the minimum of the optimal solutions of the LPs. Hence, if we fix $\epsilon$ to $\hat{\epsilon}=\varepsilon/2$, no $\mathbf{x}\in\Delta_{m}$ satisfies the constraints in the above LP for any $k\in[n]\setminus\{1,2\}$. This means that either $k$ is not a best response against $\mathbf{x}$ (Constraints (48b) and (48c)), or $k$ is a best response but the payoff of $(\mathbf{x},k)$ is lower than the highest payoff attainable by committing to a strategy in $P_{1}$ or $P_{2}$ (Constraint (48d)). In summary, $k$ cannot an SSE response in $\mathcal{G}_{d_{1}^{*}-\hat{\epsilon},d_{2}^{*}-\hat{\epsilon}}$, so only action $1$ or $2$ can be SSE responses. The same argument showing (46) and (47) also shows that only action $1$ or $2$ can be SSE responses of $\mathcal{G}_{d_{1},d_{2}}$ for all $d_{1}\in[d_{1}^{*}-\hat{\epsilon},d_{1}^{*}]$ and $d_{2}\in[d_{1}^{*}-\hat{\epsilon},d_{2}^{*}]$, so Condition (i) follows. ∎

We conclude the proof by showing that a desired $\hat{\epsilon}$ can be computed in polynomial time. Note that we cannot hope to solve the LPs defined in (48) to obtain $\hat{\epsilon}$ since we do not know $u^{L}$. Nevertheless, since Condition (i) holds for $\hat{\epsilon}<\varepsilon$ as we argued above, we can use binary search to find $\hat{\epsilon}$ from the interval $[0,\epsilon^{\prime}]$: we maintain a candidate value $\epsilon$, query the oracle $\mathcal{A}_{\textnormal{SSE}}$ to check if action $1$ or $2$ is an SSE response of $\mathcal{G}_{d_{1}^{*}-\epsilon,d_{2}^{*}-\epsilon}$, and halve $\epsilon$ if they are not. The algorithm will terminate when $\epsilon<\varepsilon$, which takes polynomial time as the bit-size of $\varepsilon$ is bounded from above by a polynomial in the size of the representation of the LPs defined in (48). ∎

E.3 Proof of Section 5.2.1

\thmsecondrefpairBRcorrespondence

Proof.

Using Section 5.2.1, we first find $\hat{\epsilon}$ satisfying the conditions stated in the lemma. Let $d_{1}=d_{1}^{*}-\hat{\epsilon}$ and $d_{2}=d_{2}^{*}-\hat{\epsilon}$. Since $d_{1}<d_{1}^{*}$ and $d_{2}<d_{2}^{*}$, we have $V_{d_{1},d_{2}}^{L}(1)<M_{\{1,2\}}$ and $V_{d_{1},d_{2}}^{L}(2)<M_{\{1,2\}}$.

If both actions $1$ and $2$ are SSE responses of $\mathcal{G}_{d_{1},d_{2}}$, then we are done: picking arbitrary $\mathbf{x}\in\max_{\mathbf{x}^{\prime}\in P_{1}(d_{1},d_{2})}\mathbf{a}_{1}\cdot\mathbf{x}^{\prime}$ and $\mathbf{y}\in\max_{\mathbf{y}^{\prime}\in P_{2}(d_{1},d_{2})}\mathbf{a}_{2}\cdot\mathbf{y}^{\prime}$ gives two SSEs $(\mathbf{x},1)$ and $(\mathbf{y},2)$; we have $u^{L}(\mathbf{x},1)=u^{L}(\mathbf{y},2)=V_{d_{1},d_{2}}^{L}(2)<M_{\{1,2\}}$.

If action $1$ is not an SSE response of $\mathcal{G}_{d_{1},d_{2}}$, we have $V_{d_{1},d_{2}}^{L}(1)<V_{d_{1},d_{2}}^{L}(2)$. Let

[TABLE]

According to Condition (ii) of Section 5.2.1, we have $V_{d,d_{2}}^{L}(1)\geq V_{d,d_{2}}^{L}(2)$ if and only if $d\geq d_{1}^{\prime}$. This means that action $1$ is an SSE response of $\mathcal{G}_{d,d_{2}}$ if and only if $d\geq d_{1}^{\prime}$, and we can use oracle $\mathcal{A}_{\textnormal{ER}}$ and binary search to pin down $d_{1}^{\prime}$. Meanwhile $V_{d_{1}^{\prime},d_{2}}^{L}(1)=V_{d_{1}^{\prime},d_{2}}^{L}(2)$, so both actions $1$ and $2$ are SSE responses when $d=d^{\prime}_{1}$. Similarly to the first case, where both actions are SSE responses of $\mathcal{G}_{d_{1},d_{2}}$, we can obtain a desired reference pair.

Similarly, if action $2$ is not an SSE response of $\mathcal{G}_{d_{1},d_{2}}$, then we search for a number

[TABLE]

Both actions $1$ and $2$ are SSE responses of $\mathcal{G}_{d_{1},d_{2}^{\prime}}$, and a desired reference pair can be obtained accordingly. ∎

E.4 Querying by Using Payoff Matrices

The above algorithm uses BR-correspondences in the queries. We next show how to transform the algorithm to one that uses payoff-based queries, thereby solving our problem completely. Using a similar approach, we first define the following payoff matrix parameterized by two numbers $d_{1}<d_{1}^{*}$ and $d_{2}<d_{2}^{*}$. We will show that the BR-correspondence of this payoff matrix functions equivalently to $f_{d_{1},d_{2}}$ defined in (23).

{boxedtext}

For every $\mathbf{x}\in\Delta_{m}$, let

[TABLE]

where $\mu$ is an arbitrary proper cover of ${\{1,2\}}$; and $\mathbf{b}_{j},c_{j}\in\mathbb{R}$ and $\mathbf{z}_{j}\in\Delta_{m}$ are functions of $d_{1}$ and $d_{2}$, defined as follows.666We omit the dependencies of these parameters on $d_{1}$ and $d_{2}$ in the notation to simplify the presentation.

• •

First, for each $j\in{\{1,2\}}$, define

[TABLE]

where for $j\in{\{1,2\}}$:

[TABLE]

• •

Let $K_{1}=[n]\setminus{\{1,2\}}$ and $K_{2}=[n]\setminus\{2\}$. We define

[TABLE]

and let $\mathbf{z}_{1}$ and $k_{1}$ be an arbitrary solution corresponding to the above maximum value, i.e.,

[TABLE]

Note that according to (49), $\tilde{u}_{d_{1},d_{2}}^{F}(\mathbf{z}_{1},k)=\mu(\mathbf{z}_{1},k)$ for all $k\in K_{1}$, so the above definitions in fact only rely on $\mu$. Furthermore, let

[TABLE]

where we use

[TABLE]

for $j\in{\{1,2\}}$, i.e., it is a value strictly larger than the maximum gap between the payoff parameters; in addition, for all $k\in[n]$, we use

[TABLE]

• •

$c_{2}$, $\mathbf{z}_{2}$, and $k_{2}$ are defined analogously by changing the labels in the above definitions from $1$ to $2$. Note that since $K_{2}=K_{1}\cup\{1\}$, these parameters also depend on $\tilde{u}_{d_{1},d_{2}}^{F}(\mathbf{x},1)=\mathbf{b}_{1}\cdot(\mathbf{x}-\mathbf{z}_{1})+c_{1}$, with $c_{1}$ and $\mathbf{z}_{1}$ defined above.

With the above definitions, we show several properties of $\tilde{u}_{d_{1},d_{2}}^{F}$ in the subsequent lemmas. We extend the notation in Section 5.2.1: hereafter, we let $\tilde{f}_{d_{1},d_{2}}$ be the BR-correspondence of $\tilde{u}_{d_{1},d_{2}}^{F}$, let $\widetilde{\mathcal{G}}_{d_{1},d_{2}}\coloneqq(u^{L},\tilde{u}_{d_{1},d_{2}}^{F})$, and let $\widetilde{V}_{d_{1},d_{2}}^{L}(j)=\max_{\mathbf{x}\in\tilde{f}_{d_{1},d_{2}}^{-1}(j)}u^{L}(\mathbf{x},j)$.

We make several observations about $\tilde{f}_{d_{1},d_{2}}$ in the following lemmas. The first lemma Figure 8 characterizes the structure of $\tilde{f}_{d_{1},d_{2}}$ in comparison to $f_{d_{1},d_{2}}$ defined in Section 5.2.1. Figure 8 provides an illustration of this characterization.

{restatable}

lemmalmmtndrefpayofffsupset

Suppose that $d_{j}\leq d_{j}^{*}$ and $d^{\prime}_{j}\in[\bar{d}_{j},d_{j}^{*}]$ for each $j\in{\{1,2\}}$. Then the following statements hold for all $j\in{\{1,2\}}$ and $k\in[n]\setminus{\{1,2\}}$:

• (i)

$\mathbf{a}_{j}\cdot\mathbf{x}\leq d^{\prime}_{j}$ for all $\mathbf{x}\in\tilde{f}_{d_{1},d_{2}}^{-1}(j)$;

• (ii)

$\bigcup_{j\in{\{1,2\}}}f_{d_{1},d_{2}}^{-1}(j)\subseteq\bigcup_{j\in{\{1,2\}}}\tilde{f}_{d_{1},d_{2}}^{-1}(j)$; and

• (iii)

$f_{d^{\prime}_{1},d^{\prime}_{2}}^{-1}(k)\subseteq\tilde{f}_{d_{1},d_{2}}^{-1}(k)\subseteq f_{d_{1},d_{2}}^{-1}(k)$.

Proof.

We prove each of the statements.

Claim (i).

Suppose that $\mathbf{a}_{j}\cdot\mathbf{x}>d^{\prime}_{j}$. We argue that $\mathbf{x}\notin\tilde{f}_{d_{1},d_{2}}^{-1}(j)$ to prove the claim.

Indeed, we have $\mathbf{a}_{j}\cdot(\mathbf{x}-\mathbf{z}_{j})>0$ and hence,

[TABLE]

where we used the fact that $\tilde{u}_{d_{1},d_{2}}^{F}(\mathbf{z}_{j},k_{j})=c_{j}$ according to the definitions of $\mathbf{z}_{j}$, $k_{j}$ and $c_{j}$. Hence, $\mathbf{x}\notin\tilde{f}_{d_{1},d_{2}}^{-1}(j)$, and Claim (i) holds.

Claim (ii).

Pick arbitrary $j\in{\{1,2\}}$ and $\mathbf{x}\in P_{j}(d_{1},d_{2})$. We argue that $\tilde{u}_{d_{1},d_{2}}^{F}(\mathbf{x},j)>\tilde{u}_{d_{1},d_{2}}^{F}(\mathbf{x},k)$, which implies Claim (ii) immediately.

Indeed, since $\mathbf{x}\in P_{j}(d_{1},d_{2})$, by definition, we have $\mathbf{a}_{j}\cdot\mathbf{x}\leq d_{j}$. Hence,

[TABLE]

for all $k\in[n]\setminus{\{1,2\}}$, where we used the fact that

[TABLE]

Claim (iii).

Consider the BR-correspondence $h:\Delta_{m}\to 2^{[n]\setminus{\{1,2\}}}$ of $\mu$. For any $d_{1}$ and $d_{2}$, according to the construction of $f_{d_{1},d_{2}}$ and $\tilde{f}_{d_{1},d_{2}}$, we have

[TABLE]

where

[TABLE]

We argue that

[TABLE]

to complete the proof.

Indeed, we have

[TABLE]

where $\operatorname{cl}(\cdot)$ denotes the closure of a set. Since (56) and (57) are strictly satisfied, we also have

[TABLE]

According to Claims (i) and (ii) we proved above,

[TABLE]

Hence, (58) holds, and this proves Claim (iii). ∎

The next lemma presents a result similar to Section E.2. Regarding Figure 8, the result indicates that $\mathbf{z}_{j}$ indeed hits the line $\mathbf{a}_{j}\cdot\mathbf{x}=\bar{d}_{j}$ whenever $Z_{j}\neq\emptyset$.

{restatable}

lemmalmmtndrefhatuL

Suppose that $j\in{\{1,2\}}$ and $Z_{j}\neq\emptyset$. Then $\mathbf{z}_{j}\in\tilde{f}_{d_{1},d_{2}}^{-1}(j)$ and $\widetilde{V}_{d_{1},d_{2}}^{L}(j)=\gamma_{j}\cdot\bar{d}_{j}+\beta_{j}$.

Proof.

It suffices to prove that $\mathbf{z}_{j}\in\tilde{f}_{d_{1},d_{2}}^{-1}(j)$. Indeed, this implies that

[TABLE]

given that $\mathbf{a}_{j}\cdot\mathbf{z}_{j}=\bar{d}_{j}$ by definition; Moreover, according to Figure 8, $\mathbf{a}_{j}\cdot\mathbf{x}\leq\bar{d}_{j}$ for all $\mathbf{x}\in\tilde{f}_{d_{1},d_{2}}^{-1}(j)$; hence,

[TABLE]

As a result, $\widetilde{V}_{d_{1},d_{2}}^{L}(j)=\gamma_{j}\cdot\bar{d}_{j}+\beta_{j}$.

Next, we prove that $\mathbf{z}_{j}\in\tilde{f}_{d_{1},d_{2}}^{-1}(j)$ to complete the proof. By definition (i.e., (49)),

[TABLE]

for all $k\in K_{j}$. Since $K_{2}=[n]\setminus\{1\}$, this immediately implies that action $2$ is a best response to $\mathbf{z}_{2}$, so $\mathbf{z}_{2}\in\tilde{f}_{d_{1},d_{2}}^{-1}(2)$.

To argue that $\mathbf{z}_{1}\in\tilde{f}_{d_{1},d_{2}}^{-1}(1)$, since $K_{1}=[n]\setminus{\{1,2\}}$, we need to prove in addition that $\tilde{u}_{d_{1},d_{2}}^{F}(\mathbf{z}_{1},1)\geq\tilde{u}_{d_{1},d_{2}}^{F}(\mathbf{z}_{1},2)$. Indeed, by definition, we have $\mathbf{a}_{2}\cdot\mathbf{z}_{1}\geq\bar{d}_{2}$ (i.e., see (50) and (53)), so similar to (56), we have that $\tilde{u}_{d_{1},d_{2}}^{F}(\mathbf{z}_{1},2)\leq\tilde{u}_{d_{1},d_{2}}^{F}(\mathbf{z}_{1},k_{2})$. Since $k_{2}\in K_{1}\cup\{1\}$, applying (59) then gives

[TABLE]

Consequently, $\tilde{u}_{d_{1},d_{2}}^{F}(\mathbf{z}_{1},1)\geq\tilde{u}_{d_{1},d_{2}}^{F}(\mathbf{z}_{1},k)$ for all $k\in[n]$, so $\mathbf{z}_{1}\in\tilde{f}_{d_{1},d_{2}}^{-1}(1)$. This completes the proof. ∎

Finally, the following lemma presents a result similar to Section 5.2.1. The result is key to the first step of our approach, where we restrict the search space in a one-dimensional interval.

{restatable}

lemmalmmtndrefpayoffotSSEresponse

Suppose that $\epsilon^{\prime}>0$ is the optimal value of LP (44). It holds for all $\delta_{1}\in[d_{1}^{*}-\epsilon^{\prime},d_{1}^{*}]$ and $\delta_{2}\in[d_{2}^{*}-\epsilon^{\prime},d_{2}^{*}]$ that:

• (i)

$Z_{j}(\delta_{1},\delta_{2})\neq\emptyset$ for both $j\in{\{1,2\}}$; and

• (ii)

if at least one of $j\in{\{1,2\}}$ is an SSE response of $\widetilde{\mathcal{G}}_{\delta_{1},\delta_{2}}$, then the same holds for $\widetilde{\mathcal{G}}_{\delta^{\prime}_{1},\delta^{\prime}_{2}}$, with arbitrary $\delta^{\prime}_{1}\in[\bar{\delta}_{1},d_{1}^{*}]$ and $\delta^{\prime}_{2}\in[\bar{\delta}_{2},d_{2}^{*}]$, where $\bar{\delta}_{j}=(\delta_{j}+d_{j}^{*})/2$.

Proof.

First, consider Condition (i). If $\epsilon\leq\epsilon^{\prime}$, we have $\bar{\delta}_{j}\in[\delta_{j},d_{j}^{*}]\subseteq[d_{j}^{*}-\epsilon^{\prime},d_{j}^{*}]$. Hence, according to Section 5.2.1, we have $P_{j}(\bar{\delta}_{1},\bar{\delta}_{2})\neq\emptyset$. By definition,

[TABLE]

It then suffices to argue that there exists $\mathbf{x}\in P_{j}(\bar{\delta}_{1},\bar{\delta}_{2})$ such that $\mathbf{a}_{j}\cdot\mathbf{x}=\bar{\delta}_{j}$. Indeed, now that $P_{j}(\bar{\delta}_{1},\bar{\delta}_{2})\neq\emptyset$, applying Section E.2, we get that there exists $\mathbf{x}\in P_{j}(\bar{\delta}_{1},\bar{\delta}_{2})$ such that $u^{L}(\mathbf{x},j)=\gamma_{j}\cdot\bar{\delta}_{j}+\beta_{j}$, which immediately implies that $\mathbf{a}_{j}\cdot\mathbf{x}=\bar{\delta}_{j}$.

Now consider Condition (ii). Now that Condition (i) holds, we have $Z_{j}(\delta_{1},\delta_{2})\neq\emptyset$ and $Z_{j}(\delta^{\prime}_{1},\delta^{\prime}_{2})\neq\emptyset$ for both $j\in{\{1,2\}}$. According to Section E.4, we then have

[TABLE]

where $\bar{\delta}^{\prime}_{j}=(\delta^{\prime}_{j}+d_{j}^{*})/2$.

Moreover, since some $j\in{\{1,2\}}$ is an SSE response of $\widetilde{\mathcal{G}}_{\delta_{1},\delta_{2}}$, by definition, this means that

[TABLE]

for all $k\in[n]\setminus{\{1,2\}}$. According to Claim (iii) of Figure 8 (and note that $\delta^{\prime}_{j}\in[\bar{\delta}_{j},d_{j}^{*}]$), we have $\tilde{f}_{\delta^{\prime}_{1},\delta^{\prime}_{2}}^{-1}(k)\subseteq f_{\delta^{\prime}_{1},\delta^{\prime}_{2}}^{-1}(k)\subseteq\tilde{f}_{\delta_{1},\delta_{2}}^{-1}(k)$. As a result, $\tilde{f}_{\delta^{\prime}_{1},\delta^{\prime}_{2}}^{-1}(k)\subseteq\tilde{f}_{\delta_{1},\delta_{2}}^{-1}(k)$, which implies that

[TABLE]

Combining (E.4)–(62), we get that

[TABLE]

for all $k\in[n]\setminus{\{1,2\}}$. Therefore, only actions $1$ and $2$ can be SSE responses of $\widetilde{\mathcal{G}}_{\delta_{1},\delta_{2}}$. ∎

With the above results in hand, we summarize our approach in the following theorem, which completes our task. The approach is similar to that used to prove Section 5.2.1, where we first search for values for $d_{1}$ and $d_{2}$ that make at least one of action $1$ and $2$ an SSE response. Then we further expand the best response region (see Figure 8) corresponding to the action that is not yet an SSE response while fixing the region of the other action, until both actions are SSE responses.

We remark that despite the similarity, one difference between the approaches in Figure 7 and Section 5.2.1 is that once we find a position where one of $j\in{\{1,2\}}$, say action $1$, is an SSE response, we cannot simply fix $d_{1}$ and and increase $d_{2}$ in the hope of finding another position where both of them are SSE responses. This is because the way $\tilde{u}_{d_{1},d_{2}}^{F}$ is constructed does not ensure that $\tilde{f}_{d_{1},d_{2}}^{-1}(2)\subseteq\tilde{f}_{d_{1},d^{\prime}_{2}}^{-1}(2)$ for any $d_{2}\leq d^{\prime}_{2}$ as our choice of $\mathbf{z}_{j}$ is arbitrary from the set $Z_{j}$.777It might be possible to ensure this by using more carefully selected $\mathbf{z}_{j}$. However, this introduces other complexities in the approach, as well as the presentation of it. Consequently, we can not ensure that $\tilde{f}_{d_{1},d_{2}}^{-1}(k)\supseteq\tilde{f}_{d_{1},d^{\prime}_{2}}^{-1}(k)$ for the other actions $k\in[n]\setminus$, which means $\widetilde{V}_{d_{1},d_{2}}^{L}(k)$ may grow when we increase $d_{2}$, potentially exceeding $\widetilde{V}_{d_{1},d_{2}}^{L}(1)$ and $\widetilde{V}_{d_{1},d_{2}}^{L}(2)$, in which case $1$ and $2$ cannot be SSE responses anymore. To resolve this issue, once we find a position $(d_{1},d_{2})$ such that one of $j\in{\{1,2\}}$ is an SSE response, our approach is to first “jump” to $(\bar{d}_{1},\bar{d}_{2})$. This ensures that $\tilde{f}_{d_{1},d_{2}}^{-1}(k)\supseteq\tilde{f}_{d_{1},d^{\prime}_{2}}^{-1}(k)$ for all $k\in[n]\setminus{\{1,2\}}$ according to the structure of $\tilde{f}_{d_{1},d_{2}}$ we demonstrated in Figure 8 (and Figure 8).

\thmsecondrefpayoffquery

Proof.

The algorithm is similar to the approach used in Section 5.2.1 and proceeds as follows.

• •

Step 1. Let $\epsilon^{\prime}$ be the optimal value of LP (44). We first search for an $\epsilon\in(0,\epsilon^{\prime}]$ such that one of $j\in{\{1,2\}}$ is an SSE response of $\widetilde{\mathcal{G}}_{d_{1}^{*}-\epsilon,d_{2}^{*}-\epsilon}$.

Specifically, we use binary search to find such an $\epsilon$: we start with the candidate value $\varepsilon=\epsilon^{\prime}$ and halve $\varepsilon$ if none of $j\in{\{1,2\}}$ is an SSE response of $\widetilde{\mathcal{G}}_{d_{1}^{*}-{\varepsilon},d_{2}^{*}-{\varepsilon}}$. Note that by Section E.2, $\epsilon^{\prime}>0$, so applying Section E.4 we get that $Z_{j}(d_{1}^{*}-\varepsilon,d_{2}^{*}-\varepsilon)\neq\emptyset$, which means that $\widetilde{\mathcal{G}}_{d_{1}^{*}-{\varepsilon},d_{2}^{*}-{\varepsilon}}$ is well defined.

Moreover, the binary search procedure will terminate in polynomial time. This is due to the following fact. Let $\hat{\epsilon}$ be a number satisfying the conditions stated in Section 5.2.1, which according to the same lemma exists and has a polynomial bit-size. For any $\varepsilon\leq\hat{\epsilon}$, we can show that at least one of $j\in{\{1,2\}}$ must be an SSE response of $\widetilde{\mathcal{G}}_{d_{1}^{*}-{\varepsilon},d_{2}^{*}-{\varepsilon}}$, so it follows immediately that the binary search procedure terminates in polynomial time. Indeed, according to Section 5.2.1, some $j\in{\{1,2\}}$ is an SSE response of ${\mathcal{G}}_{d_{1}^{*}-{\varepsilon},d_{2}^{*}-{\varepsilon}}$, which means that

[TABLE]

We can show that the same holds with respect to $\widetilde{V}_{d_{1}^{*}-\varepsilon,d_{2}^{*}-\varepsilon}^{L}$. Specifically, it suffices to show that

[TABLE]

(64) follows directly by Figure 8, i.e., $\tilde{f}_{d_{1}^{*}-\varepsilon,d_{2}^{*}-\varepsilon}^{-1}(k)\subseteq f_{d_{1}^{*}-\varepsilon,d_{2}^{*}-\varepsilon}^{-1}(k)$.

To see that (63) holds, note that since $Z_{j}(d_{1}^{*}-{\varepsilon},d_{2}^{*}-{\varepsilon})\neq\emptyset$, by Section E.4, we have

[TABLE]

Since $\varepsilon>0$, it follows that

[TABLE]

where the second inequality follows by the fact that $\mathbf{a}_{j}\cdot\mathbf{x}\leq d_{j}^{*}-\varepsilon$ for all $\mathbf{x}\in f_{d_{1}^{*}-\varepsilon,d_{2}^{*}-\varepsilon}^{-1}(j)=P_{j}(d_{1}^{*}-\varepsilon,d_{2}^{*}-\varepsilon)$.

• •

Step 2. Let $\epsilon$ be the outcome of Step 1. Let $d_{j}=d_{j}^{*}-{\epsilon}$ and $\bar{d}_{j}=(d^{*}+d_{j})/2$ for $j\in{\{1,2\}}$. We search for $\delta_{j}\in[\bar{d}_{j},d_{j}^{*})$ such that both actions $1$ and $2$ are SSE responses of $\widetilde{\mathcal{G}}_{\delta_{1},\delta_{2}}$.

Indeed, if both $1$ and $2$ are SSE responses of $\widetilde{\mathcal{G}}_{\bar{d}_{1},\bar{d}_{2}}$, then we are done. Otherwise, suppose that only action $1$ is an SSE response. Then we fix $\delta_{1}=\bar{d}_{1}$ and search for a value $\delta_{2}$ such that

[TABLE]

Given Section E.4, and now that $Z_{j}(\delta_{1},\delta_{2})\neq\emptyset$ according to Section E.4, only when $\delta_{2}^{\prime}=\delta_{2}$, we have $\widetilde{V}_{\delta_{1},\delta_{2}^{\prime}}^{L}(1)=\widetilde{V}_{\delta_{1},\delta_{2}^{\prime}}^{L}(2)$, in which case both actions are SSE responses of $\widetilde{\mathcal{G}}_{\delta_{1},\delta_{2}^{\prime}}$. Hence, with access to $\mathcal{A}_{\textnormal{ER}}$, we can use binary search to find out $\delta_{2}$: whenever action $2$ is not an SSE response of $\widetilde{\mathcal{G}}_{\delta_{1},\delta_{2}^{\prime}}$, we know that $\delta^{\prime}_{2}<\delta_{2}$; and whenever action $1$ is not, we know that $\delta^{\prime}_{2}>\delta_{2}$. Similarly, if only action $2$ is an SSE response of $\widetilde{\mathcal{G}}_{\bar{d}_{1},\bar{d}_{2}}$, we fix $\delta_{2}=\bar{d}_{2}$ and search for $\delta_{1}=(\widetilde{V}_{\bar{d}_{1},\bar{d}_{2}}^{L}(2)-\beta_{1})/\gamma_{1}$.

• •

Step 3. Finally, consider $\tilde{u}_{\delta_{1},\delta_{2}}^{F}$ and let $\mathbf{z}_{1}$ and $\mathbf{z}_{2}$ be the parameters corresponding to this payoff matrix. It can be verified that $(\mathbf{z}_{1},1)$ and $(\mathbf{z}_{2},2)$ are a reference pair such that $u^{L}(\mathbf{z}_{1},1)=u^{L}(\mathbf{z}_{2},2)<M_{\{1,2\}}$.

Indeed, according to Section E.4, $j$ is a best response of $\mathbf{z}_{j}$ for both $j\in{\{1,2\}}$; moreover,

[TABLE]

where $\bar{\delta}_{j}\coloneqq(d_{j}^{*}+\delta_{j})/2<d_{j}^{*}$. Now that both actions are SSE responses, we have $\widetilde{V}_{\delta_{1},\delta_{2}}^{L}(1)=\widetilde{V}_{\delta_{1},\delta_{2}}^{L}(2)$. Hence, $u^{L}(\mathbf{z}_{1},1)=u^{L}(\mathbf{z}_{2},2)<M_{\{1,2\}}$. ∎