Robust Commitments and Partial Reputation

TL;DR

This paper investigates how a leader in Stackelberg games can optimally shape their observable commitment to maximize payoff, proposing robust strategies that perform well under observational uncertainty.

Contribution

It introduces observation-robust, polynomial-time algorithms for leader commitments that approximate and potentially surpass traditional Stackelberg payoffs in non-zero-sum settings.

Findings

Robust commitment strategies improve leader payoff under observational uncertainty.

Traditional mixed Stackelberg commitments are not robust to observation errors.

Proposed methods are computationally efficient and applicable to security and economic scenarios.

Abstract

Agents rarely act in isolation -- their behavioral history, in particular, is public to others. We seek a non-asymptotic understanding of how a leader agent should shape this history to its maximal advantage, knowing that follower agent(s) will be learning and responding to it. We study Stackelberg leader-follower games with finite observations of the leader commitment, which commonly models security games and network routing in engineering, and persuasion mechanisms in economics. First, we formally show that when the game is not zero-sum and the vanilla Stackelberg commitment is mixed, it is not robust to observational uncertainty. We propose observation-robust, polynomial-time-computable commitment constructions for leader strategies that approximate the Stackelberg payoff, and also show that these commitment rules approximate the maximum obtainable payoff (which could in general be greater than the Stackelberg payoff).

Equations

$${\mathcal{K}}^{\ast }(\mathbf{x}):={\arg \max }_{j\in [n]}⟨\mathbf{x},{\mathbf{b}}_j⟩.$$

$$j^{\ast }(\mathbf{x}):={\arg \max }_{j\in {\mathcal{K}}^{\ast }(\mathbf{x})}⟨\mathbf{x},{\mathbf{a}}_j⟩.$$

$$f_{\infty }(\mathbf{x}):=⟨\mathbf{x},{\mathbf{a}}_{k^{\ast }}⟩.$$

$$f_{\infty }^{\ast }:=\underset{\mathbf{x}\in {\Delta }_m}{\max }{f}_{\infty }(\mathbf{x}).$$

$$j^{\ast }({\hat{\mathbf{X}}}_N).$$

$$f_N(\mathbf{x}):=\mathbb{E}\left[⟨\mathbf{x},{\mathbf{a}}_{j^{\ast }({\hat{\mathbf{X}}}_N)}⟩\right]$$

$$f_N^{\ast }:=\underset{\mathbf{x}\in {\Delta }_m}{\max }{f}_N(\mathbf{x})$$

$$\{\begin{array}{l}f(p;1):=p\text{ if follower best responds with strategy 1 }\\ f(p;2):=1-p\text{ if follower best responds with strategy 2 }\\ f(p;3):=3-4p\text{ if follower best responds with strategy 3 }\end{array}$$

$$f_{\infty }(p)=\min \{f(p;1),f(p;2),f(p;3)\}.$$

$$f_{\infty }^{\ast }=\underset{p\in [0,1]}{\max }{f}_{\infty }(p)=1/2,$$

$$f_1(1/2)$$

$$=1/2\times 1/2+1/2\times 1=3/4.$$

$$f_N(1/2)-f_{\infty }^{\ast }$$

$$=1/2\mathrm{Pr}[{\hat{P}}_N-1/2>1/6]$$

$$\le \exp \{-N{D}_{\mathsf{KL}}(2/3\Vert 1/2)\}$$

$$f_{\infty }(p)=\{\begin{array}{l}p\text{ if }p\le 1/2\\ -p\text{ if }p>1/2.\end{array}$$

$$f_N(1/2)$$

$$=1/2\times 1/2-1/2\times 1/2=0.$$

$$f_{\infty }(p)=\{\begin{array}{l}p\text{ if }p\le 1/2\\ 1/2-p\text{ if }1/2<p\le 5/7\\ 3-4p\text{ if }p>5/7.\end{array}$$

$$f_N(1/2)$$

$$=(1/2)(1/2)+\mathrm{Pr}[{\hat{P}}_N\ge 5/7](1)$$

$$\le 1/4+1/2\exp \{-N{D}_{\mathsf{KL}}(5/7\Vert 1/2)\}.$$

$$f_N(p_{\infty }^{\ast })\le f_{\infty }^{\ast }-C\left(\varphi (\sqrt{N}{C}^{\prime })-\frac{1}{2}-\frac{C^{\prime }}{\sqrt{N}}\right)+\exp \{-N{C}^{\prime \prime 2}\}$$

$$f_{\infty }^{\ast }-f_N({\mathbf{x}}_N)$$

$$f_N^{\ast }$$

$${\mathcal{K}}_{\mathsf{alt}}^{\ast }(\mathbf{x}):={\mathcal{K}}^{\ast }(\mathbf{x})-\{j^{\ast }\}.$$

$$f_{max}$$

$$f_{min}$$

$${\mathcal{R}}_{j_{\infty }^{\ast }}$$

$${\mathcal{R}}_{j_{\mathsf{alt}}}$$

$$f_N(p_{\infty }^{\ast })$$

$$=\mathrm{Pr}\left[{\hat{P}}_N\in (p^{-},p_{\infty }^{\ast }]\right]f_{\infty }^{\ast }+\mathrm{Pr}\left[{\hat{P}}_N\in (p_{\infty }^{\ast },p^{+}]\right]f^{(2)}+\mathrm{Pr}\left[{\hat{P}}_N\in [0,p^{-}]\cup (p^{+},1]\right]f_{max}$$

$$=f_{\infty }^{\ast }-\underset{T_1(N)}{\underbrace{\mathrm{Pr}\left[{\hat{P}}_N\in (p_{\infty }^{\ast },p^{+}]\right]}}(f_{\infty }^{\ast }-f^{(2)})+\underset{T_2(N)}{\underbrace{\mathrm{Pr}\left[{\hat{P}}_N\in [0,p^{-}]\cup (p^{+},1]\right]}}(f_{max}-f_{\infty }^{\ast }).$$

$$T_2(N)$$

$$=\mathrm{Pr}\left[{\hat{P}}_N\in [0,p^{-}]\right]+\mathrm{Pr}\left[{\hat{P}}_N\in (p^{+},1]\right]$$

$$\le \exp \{-2N(p_{\infty }^{\ast }-p^{-}{)}^2\}+\exp \{-2N(p^{+}-p_{\infty }^{\ast }{)}^2\}.$$

Full text

1 Introduction

Consider a selfish, rational agent (designated as the “leader”) who is interacting non-cooperatively with another selfish, rational agent (designated as the “follower”). If the agents are interacting simultaneously, and know the game that they are playing, they will naturally play a Nash equilibrium(a) of the two-player game. This is the traditionally studied solution concept in game theory. Now, let’s say that the leader has the ability to reveal its strategy in advance – in the form of a mixed strategy commitment, and the follower has the ability to observe this commitment and respond to it. The optimal strategy for the leader now corresponds to the Stackelberg equilibrium of the two-player leader-follower game. The Stackelberg solution concept enjoys application in several engineering settings where commitment is the natural framework for the leader, such as security [PPM*+*08], network routing [Rou04] and law enforcement [MS17]. The solution concept is interpreted in a broader sense as the ensuing strategy played by a patient leader that wishes to build a reputation by playing against an infinite number of myopic followers [KW82, MR82, FL89, FL92]. Crucially, one can show rigorously that the leader will benefit significantly from commitment power, i.e. its ensuing Stackelberg equilibrium payoff is at least as much as the simultaneous equilibrium payoff [VSZ10]. Further, several mechanism design problems that involve private information revelation - this includes signalling games [CS82] and persuasion games [KG11] - can be thought of as Stackelberg games, and the optimal mechanism can be interpreted as the Stackelberg commitment111albeit sometimes with multiple followers..

But whichever interpretation one chooses, the Stackelberg solution concept assumes a very idealized setting (even over and above the assumptions of selfishness and infinite rationality) in which the mixed strategy commitment is exactly revealed to the follower. Further, the follower 100% believes that the leader will actually stick to her commitment. What happens when these assumptions are relaxed? What if the leader could only demonstrate her commitment in a finite number of interactions – how would she modify her commitment to maximize payoff, and how much commitment power would she continue to enjoy? Is she even incentivized to help the follower estimate her commitment effectively?

What changes in a finite-interaction regime is that the follower only observes a part of the leader behavioral history and needs to learn about the leader’s strategic behavior – to the extent that he is able to respond as optimally as possible. By restricting our attention to finitely repeated play, we arrive at a problem setting that is fairly general: these follower agents in general will not know about the preferences of the leader agent. When provided with historical context, we assume that they will use it rather than ignore it. A broad umbrella of problems that has received significant attention in the machine learning literature is learning of strategic behavior from samples of play; for example, learning the agent’s utility function through inverse reinforcement learning [ZMBD08], learning the agent’s level of rationality [WZB13], and inverse game theory [KS15]. While significant progress has been made in this goal of learning strategic behavior, attention has been restricted to the passive learning setting in which the leading agent is unaware of the presence of the learner, or agnostic to the learner’s utility. In many situations, the agent himself will be invested in the outcome of the learning process. In this paper, we put ourselves in the shoes of an agent who is shaping her historical context and is aware of the learner’s presence as well as preferences, and study her choice of optimal strategy222It is worth mentioning the recent paradigm of cooperative inverse reinforcement learning [HMRAD16] which studies the problem of agent investment in principal learning where the incentives are not completely aligned, but the setting is cooperative. In contrast, we focus on non-cooperative settings.. As we will see, the answer will depend on her utility function itself, as well as what kind of response she is able to elicit from the learner.

1.1 Related work

The Stackelberg solution concept is used in the engineering and economics literature to model a number of scenarios. For one, the Stackelberg security game is played between a defender, who places different utility levels on different targets to be protected and accordingly uses her resources to defend some subset of targets; and an attacker, who observes the defender strategy and wishes to attack some of the targets depending on whether he thinks they are left open as well as how much he values those targets. Stackelberg games can also be modeled with a single leader and multiple followers, such as in computer network routing applications [Rou04]. Many mechanism design problems involve computing an optimal mechanism to commit to, or an optimal way of revealing private information - this includes auctions and, more recently, Bayesian persuasion mechanisms [KG11].

Economists have established an important link between the Stackelberg solution concept and the asymptotic limit of reputation building. Reputation effects were first observed in the chain-store paradox, a firm-competition game where an incumbent firm would often deviate from Nash equilibrium behavior and play its aggressive Stackelberg (pure, in this case) strategy [Sel78]. Theoretical justification was provided for this particular application [KW82, MR82] by modeling a $(N+1)$-player interaction between a leader and multiple followers, and studying the Nash equilibrium of an ensuing game as $N\to\infty$. It was shown that the leader would play its pure Stackelberg strategy333This is clearly more restrictive than the mixed strategy Stackelberg solution concept, and not necessarily advantageous over Nash, but it turns out to be so in the firm competition case. in the Bayes-Nash equilibrium of this game endowed with a common prior on the leader’s payoff structure444A more nuanced model considered a Bayesian prior over a leader being constrained to play its pure Stackelberg strategy as opposed to unconstrained play.. This model was generalized to such leader-follower ensembles for a general two-player game, and considering the possibility of mixed-strategy reputation, still retaining the asymptotic nature of results [FL89, FL92]. The “first-player” advantage, and the entire Stackelberg solution concept, rely on an important assumption: that the commitment is perfectly revealed to the follower. This is usually not the case: in security games, the attacker will usually observe a finite number of deployments of the defender’s resource, as opposed to the allocation strategy itself (which is often mixed). In theoretical models for Bayesian persuasion, the persuader conveys a conditional distribution on her signal given the privately observed state of the world - but what will be practically observed is her history, and thus realizations of the signal, not the distribution itself. In all of these models, the leader establishes her reputation only partially, and the manifestation of the revelation is itself random. It is natural to ask how she should plan to optimally reveal her information under this constraint.

The idea of a robust solution concept in game theory is certainly not new. The concept of trembling-hand-perfect-equilibrium [Sel75] explicitly studies how robust mixed-strategy Nash equilibria are to slight perturbations in the mixtures themselves, and a similar concept was proposed for Stackelberg [VDH97]. Another solution concept, quantal-response-equilibrium [MP95], studies agents that are boundedly rational, an orthogonal but important source of uncertainty in response. In the Stackelberg setting, it was noted that robust commitments exist that preserve the Stackelberg guarantee for small enough amounts of noise in the commitment; however, this is still an asymptotic perspective and does not directly help us answer our key computational questions: can we construct a robust commitment efficiently when the game is multi-dimensional, and does the leader want to use the noise to reveal or obfuscate her commitment?

The problem of computing the optimal commitment under finitely limited observability corresponds to a robust optimization problem that is, in fact, NP-hard [AKK*+*12, SAY*+*12]; so directly reasoning about the optimal commitment is not easy. Whether there exists a polynomial-time approximation scheme for this problem was also unclear. A duo of papers [AKK*+*12, SAY*+*12] considered a model of full-fledged observational uncertainty with a Bayesian prior and posterior update based on samples of behavior, and proposed heuristic algorithmic techniques to compute the optimum. In fact, they also factored for quantal responses using a bounded rationality model [MP95]. This work showed through simulations that there could be a positive return over and above Stackelberg payoff. In one important piece of analytical work, the problem was also considered for the special case of zero-sum games [BHP14], and it was shown that the Stackelberg commitment itself approximated the optimal payoff. In this result, the extent of approximation actually depends on the amount of observational uncertainty itself - the results we prove for all non-zero-sum games have a similar flavor.

The problem of communication constraints in the commitment has also received a lot of interest in the recent algorithmic persuasion literature, but with quantitatively different models for the uncertainty. Communication constraints on signaling in bilateral trading games [DKQ16] and auction design [DIR14] have been studied from a compression perspective, where the leading agent can design the observation channel - while in our model the observation channel is even more constrained to be a finite number of random realizations of the mixed commitment. Further, in many of these settings, the principal is naturally incentivized to reveal the private information and the problem primarily becomes about the communication complexity, and whether the social welfare of the optimal mechanism can be approximated555These are clearly interesting algorithmic questions in themselves, especially in the case of multiple receivers and private vs public signaling [DX16], but do not directly address the questions we have raised.. In security games, the possibility of the mixed commitment being either fully observed or not observed at all has been considered [KCP11]; as well as different ways of handling the uncertainty, eg: showing that for some security games the Stackelberg and Nash equilibria coincide and observation of commitment does not matter [KYK*+*11]. Pita et al [PJT*+*10] first proposed a model for the defender (leader) to account for attacker (follower) observational uncertainty by allowing the follower to anchor [RT97] to a certain extent on its uniform prior. While they showed significant returns from using their model through extensive experiments, they largely circumvented the algorithmic and analytical challenges by not explicitly considering random samples of defender behavior, thus keeping the attacker response deterministic but shifted. Our work limits observation in the most natural way for the applications that we consider (i.e. number of samples of leader behavior), and because the manifestation of the uncertainty is itself random, our results have distinct and new implications.

1.2 Our contributions

Our main contribution is to understand the extent of reputational advantage when interaction is finite, and prescribe approximately optimal commitment rules for this regime.

We study Stackelberg leader-follower games in which a follower obtains a limited number of observations of the leader commitment. We first prove that in most non-zero-sum games the payoff of the Stackelberg commitment is not robust to even an infinitesmal amount of observational uncertainty. Therefore, the Stackelberg commitment is suboptimal in its payoff666This property has actually been proved for special examples of Stackelberg games [VDH97], but it was unclear whether it holds for all or most games.. Next, we propose robust commitment rules for leaders and show that we can approach the Stackelberg payoff as the number of observations increases. The robust commitment construction involves optimizing a tradeoff between preserving the follower best response and staying close to the ideal Stackelberg commitment, by moving the commitment a little bit into the interior of an appropriate convex polytope [CS06]. The analysis of payoff of the commitment construction is inspired by interior point convex geometry [KN12]. Finally, we show that a possible advantage for the leader from limited observability is only related to follower response mismatch, and show that this advantage is limited. Computationally speaking, the corollary is that we are able to approximate the optimal payoff through a simple construction which can be obtained in constant time from computation of the Stackelberg commitment (itself a polynomial-time operation [CS06]). Philosophically, this result implies that a leader can gain to a very limited extent by misrepresenting her commitment and eliciting a suboptimal response from the follower. We corroborate our theoretical results with simulations on illustrative examples and random ensembles of security games.

2 Problem statement

2.1 Preliminaries

We represent a two-player leader-follower game in normal form by the pair of $d\times n$ matrices $(T_{1},T_{2})$, where $T_{1}\in\mathbb{R}^{d\times n}$ denotes the leader payoff matrix and $T_{2}\in\mathbb{R}^{d\times n}$ denotes the follower payoff matrix. We denote the leader mixed strategy space by $\Delta_{d}$ (where $\Delta_{k}$ for any $k$ represents the $k$-dimensional probability simplex) and the follower mixed strategy space by $\Delta_{n}$. From now on, we define an effective dimension of a game as a number $m<d$ for which the effective payoff matrices of leader and follower respectively are $A=\begin{bmatrix}\mathbf{a}_{1}&\mathbf{a}_{2}&\ldots&\mathbf{a}_{n}\end{bmatrix}\in\mathbb{R}^{m\times n}$, $B=\begin{bmatrix}\mathbf{b}_{1}&\mathbf{b}_{2}&\ldots&\mathbf{b}_{n}\end{bmatrix}\in\mathbb{R}^{m\times n}$, and the effective set of leader strategies is given by a convex polytope $K\subseteq\Delta_{m}$777This definition is important in the context of Stackelberg security games, for which the leader strategy space looks exponential in the number of targets $m$ - but the actual manifestation of all leader strategies is in fact $m$-dimensional. In particular, a defender strategy manifests as a distribution over different targets being covered. .

We consider a setting of asymmetric private information in which the leader knows about the follower preferences (i.e. she knows the matrix $B$) while the follower does not know about the leader preferences (i.e. he possesses no knowledge of the matrix $A$)888This is an important assumption for the paper, and is in fact used in traditional reputation building frameworks. In future, we will want to better understand situations of repeated interaction where the $\infty$-level leader and $1$-level follower are both learning about one another..

With infinite experience, the well-established effect from the follower’s point of view is that the leader has established commitment, or developed a reputation, for playing according to some mixed strategy $\mathbf{x}\in K$. We denote the follower’s set of theoretically best pure-strategy responses to a mixed strategy commitment $\mathbf{x}$ by $\mathcal{K}^{*}(\mathbf{x})\subseteq[m]$. Explicitly, we have

[TABLE]

An important assumption that we make (and that has been made in the classical literature [CS06]) is the follower actually responds with the pure strategy in the set $\mathcal{K}^{*}(\mathbf{x})$ that is most beneficial to the leader999The technical reason for this tie-breaking rule is to be able to explicitly define the Stackelberg commitment as an explicit maximum - this in itself gives a subtle clue of its fragility. . That is, the follower responds with pure strategy

[TABLE]

Then, we also define best-response regions as the set of leader commitments that would elicit the pure strategy response $j$ from the follower, i.e. $\mathcal{R}_{j}:=\{\mathbf{x}\in K:j^{*}(\mathbf{x})=j\}$.

With these definitions, we can define the leader’s ideal payoff to be expected with an infinite reputation:

Definition 1**.**

A leader with an infinite reputation of playing according to the strategy $\mathbf{x}$ should expect payoff

[TABLE]

Therefore, the leader’s Stackelberg payoff is the solution to the program

[TABLE]

The argmax of this program is denoted as the Stackelberg commitment $\mathbf{x}^{*}_{\infty}$. Further, we denote the best response faced in Stackelberg equilibrium by $j^{*}:=j^{*}(\mathbf{x}^{*}_{\infty})$.

It is clear that the Stackelberg commitment is optimal for the leader under two conditions: the leader is 100% known to be committed to a fixed strategy, and the follower knows exactly the leader’s committed-to strategy. For a finite number of interactions, neither is true.

2.2 Observational uncertainty with established commitment

Even assuming that there is a shared belief in commitment, there is uncertainty. In particular, with a finite number of plays, the follower does not know the exact strategy that the leader has committed to, and only has an estimate.

Consider the situation where a leader can only reveal its commitment $\mathbf{x}$ through $N$ pure strategy plays $I_{1},I_{2},\ldots,I_{N}\text{ i.i.d }\sim\mathbf{x}$. The commitment is known (to both leader and followers) to come from a set of mixed strategies $\mathcal{X}\subseteq K$. We denote the maximum likelihood estimate of the leader’s mixed strategy, as seen by the follower, by $\mathbf{\widehat{X}}_{N}$. It is reasonable to expect, under certainty of commitment, that a “rational”101010Rational is in quotes because the follower is not necessarily using expected-utility theory (although there is an expected-utility-maximization interpetation to this estimate if the mixed strategy were uniform drawn from $\mathcal{X}$). follower would best-respond to $\mathbf{\widehat{X}}_{N}$, i.e. play the pure strategy

[TABLE]

We can express the expected leader payoff under this learning rule.

Definition 2**.**

A leader which will have $N$ plays according to the hidden strategy $\mathbf{x}$ can expect payoff in the $N$th play of

[TABLE]

against a follower that plays according to (1). The maximal payoff a leader can expect is

[TABLE]

and it acquires this payoff by playing the argmax strategy $\mathbf{x}^{*}_{N}$.

Ideally, we want to understand how close $f^{*}_{N}$ is to $f^{*}_{\infty}$, and also how close $\mathbf{x}^{*}_{N}$ is to $\mathbf{x}^{*}_{\infty}$. An answer to the former question would tell us how observational uncertainty impacts the first-player advantage. An answer to the latter question would shed light on whether the best course of action deviates significantly from Stackelberg commitment. We are also interested in algorithmic techniques for approximately computing the quantity $f^{*}_{N}$, as doing so exactly would involve solving a non-convex optimization problem.

3 Main Results

3.1 Robustness of Stackelberg commitment to observational uncertainty

A natural first question is whether the Stackelberg commitment, which is clearly optimal if the game were being played infinitely (or equivalently, if the leader had infinite commitment power and exact public commitment), is also suitable for finite play. In particular, we might be interested in evaluating whether we can do better than the baseline Stackelberg performance $f^{*}$. We show through a few paradigmatic examples that the answer can vary.

Example 1**.**

We consider a $2\times 3$ zero-sum game, represented in normal form in Figure 1(a), in which we can express the leader strategy according to the probability $p$ with which she will pick strategy $1$, and leader payoff is as follows:

[TABLE]

Since the game is zero-sum, the follower responds in a way that is worst-case for the leader. This means that we can express the leader payoff as

[TABLE]

This leader payoff structure is depicted in Figure 1(a). Therefore, we can express the Stackelberg payoff as

[TABLE]

attained at $p^{*}_{\infty}=1/2$. We wish to evaluate $f_{N}(p^{*}_{\infty})$. It was noted [BHP14] that $f_{N}(p^{*}_{\infty})\geq f^{*}_{\infty}(p^{*}_{\infty})$ by the minimax theorem, but not always clear whether strict inequality would hold (that is, if observational uncertainty gives a strict advantage). For this example, we can actually get a sizeable improvement! To see this, look at the simple example of $N=1$. Denoting $\widehat{P}=\frac{1}{N}\sum_{j=1}^{N}\mathbb{I}[I_{j}=1]$, we have

[TABLE]

The semilog plot in Figure 2 shows that this improvement persists for larger values of $N$, although the extent of improvement decreases exponentially with $N$. We can show that

[TABLE]

where $D_{\mathsf{KL}}(.\;\|\;.)$ denotes the Kullback-Leibler divergence, and the last inequality is due to Sanov’s theorem [CK11].

This shows analytically that the advantage does indeed decrease exponentially with $N$. Naturally, this is because we see a decrease in the stochasticity that elicits the more favorable follower response with action $3$ with the number of observations $N$.

Example 1 showed us how Stackelberg commitment power could be increased by stochastically eliciting more favorable responses. We now see an example illustrating that the commitment power can disappear completely.

Example 2**.**

We consider a $2\times 2$ non-zero-sum game, represented in normal form and leader payoff structure in Figure 1(b). Explicitly, the ideal leader payoff function is

[TABLE]

This is essentially the example reproduced in [BHP14], which we repeat for storytelling value. Notice that $f^{*}_{\infty}=1/2,p^{*}_{\infty}=1/2$, but the advantage evaporates with observational uncertainty. For any finite $N$, we have

[TABLE]

Remarkably, this implies that $f^{*}_{\infty}-f_{N}(p^{*}_{\infty})=1/2$ and so $\lim_{N\to\infty}f^{*}_{\infty}-f_{N}(p^{*}_{\infty})\neq 0$! This is clearly a very negative result for the robustness of Stackelberg commitment, and as a very pragmatic matter tells us that the idealized Stackelberg commitment $p^{*}_{\infty}$ is far from ideal in finite-observation settings. This example shows us a case where stochasticity in follower response is not desired, principally because of the discontinuity in the leader payoff function at $p^{*}_{\infty}$.

Example 2 displayed to the fullest the significant disadvantage of observational uncertainty. The game considered was special in that there was no potential for limited-observation gain, while in the game presented in Example 1 there was only potential for limited-observational gain. What could happen in general? Our next and final example provides an illustration.

Example 3**.**

Our final example considers a $2\times 3$ non-zero-sum game, whose normal form and leader payoff structure are depicted in Figure 1(c). The ideal leader payoff function is

[TABLE]

As in the other examples, $f^{*}_{\infty}=1/2,p^{*}_{\infty}=1/2$. Notice that this example captures both positive and negative effects of stochasticity in response. On one hand, follower response $2$ is highly undesirable (a la Example 2) but follower response $3$ is highly desirable (a la Example 1). What is the net effect? We have

[TABLE]

A quick calculation thus tells us that $f_{N}(p^{*}_{\infty})<=f^{*}_{\infty}$ if $N\geq 8$, showing that Stackelberg in fact has poor robustness for this example. Intuitively, the probability of the “bad” stochastic event remains constant while the probability of the “good” stochastic event decreases exponentially with $N$. Even more damningly, we see that $\lim_{N\to\infty}f^{*}_{\infty}-f_{N}(p^{*}_{\infty})\geq\lim_{N\to\infty}1/4-1/2\exp\{-ND_{\mathsf{KL}}(5/7\;\|\;1/2)=1/4$, again showing that the Stackelberg commitment is far from ideal. We can see the dramatic decay of leader advantage over and above Stackelberg, and ensuing disadvantage even for a very small number of observations, in Figure 3(a).

While the three examples detailed above provide differing conclusions, there are some common threads. For one, in all the examples it is the case that committing to the Stackelberg mixture $\mathbf{x}^{*}_{\infty}$ can result in the follower being agnostic between more than one response. Only one of these responses, the pure strategy $j^{*}$, is desirable for the leader. A very slight misperception in the estimation of the true value $\mathbf{x}^{*}_{\infty}$ can therefore lead to a different, worse-than-expected response and this misperception happens with a sizeable, non-vanishing probability. On the flipside, a different response could also lead to better-than-expected payoff, raising the potential for a gain over and above $f^{*}$. However, these better-than-expected responses cannot share a boundary with the Stackelberg commitment, and we will see that the probability of eliciting them decreases exponentially with $N$. The net effect is that the Stackelberg commitment is, most often, not robust – and critically, this is even the case for small amounts of uncertainty.

Our first result is a formal statement of instability of Stackelberg commitments for a general $2\times n$ game. We denote the leader probability of playing strategy $1$ by $p\in[0,1]$, and the Stackelberg commitment’s probability of playing strategy $1$ by $p^{*}_{\infty}$.

Furthermore, let $\phi(t)$ denote the CDF of the standard normal distribution $\mathcal{N}(0,1)$. We are now ready to state the result.

Theorem 1**.**

For any $2\times n$ leader-follower game in which $p^{*}_{\infty}\in(0,1)$ and $f_{\infty}(p)$ discontinuous at $p=p^{*}_{\infty}$, we have

[TABLE]

where $C,C^{\prime},C^{\prime\prime}$ are strictly positive constants depending on the parameters of the game. This directly implies the following:

For some $N_{0}>0$, we have $f_{N}(p^{*}_{\infty})<f^{*}_{\infty}$ for all $N>N_{0}$. 2. 2.

We have $\lim_{N\to\infty}f_{N}(p^{*}_{\infty})<f^{*}_{\infty}$.

The proof of Theorem 1 is contained in Section A.1. The technical ingredients in the proof are the Berry-Esseen theorem [Ber41, Ess42], used to show that the detrimental alternate responses on the Stackelberg boundary are non-vanishingly likely – and the Hoeffding bound, used to tail bound the probability of potentially beneficial alternate responses not on the boundary111111It is worth noting that a similar argument as presented here could be extended to a general $m\times n$ game, using iid random vectors instead of random variables and considering a demarcation into best-response regions as illustrated in Figure 7. We restrict attention to the $2\times n$ case for ease of exposition.

For non-robustness of Stackelberg commitment to hold, the two critical conditions for the game are that there is a discontinuity at the Stackelberg boundary, and that the Stackelberg commitment is mixed. For a zero-sum game, the first condition does not hold and the Stackelberg commitment stays robust as we saw in Example 1.

The theorem directly implies that the ideal Stackelberg payoff is only obtained for the exact case of $N=\infty$ (when the commitment is perfectly observed), and that for any value of $N<\infty$ there is a non-vanishing reduction in payoff. In the simulations in Section 4, we will see that this gap is empirically significant.

3.2 Robust commitments achieving close-to-Stackelberg performance

The surprising message of Theorem 1 is that, in general, the Stackelberg commitment $\mathbf{x}^{*}$ is undesirable. The commitment $\mathbf{x}^{*}$ is pushed to the extreme point of the best-response-region $\mathcal{R}_{j^{*}}$ to ensure optimality under idealized conditions; and this is precisely what makes it sub-optimal under uncertainty. What if we could move our commitment a little bit into the interior of the region $\mathcal{R}_{j^{*}}$ instead, such that we can get a high-probability-guarantee on eliciting the expected response, while staying sufficiently close to the idealized optimum? Our next result quantifies the ensuing tradeoff and shows that we can cleverly construct the commitment to approximate Stackelberg performance.

Theorem 2**.**

Let the best-response polytope $\mathcal{R}_{j^{*}}$ be non-empty in $\mathbb{R}^{m-1}$. Then, provided that the number of samples $N=\widetilde{\mathcal{O}}(m)$, we can construct commitment $\mathbf{x}_{N,p}$ for every $0<p<1/2$ such that

[TABLE]

Furthermore, these constructions are computable in $\mathcal{O}(1)$ time with knowledge of the Stackelberg commitment $\mathbf{x}^{*}_{\infty}$. (The $\widetilde{\mathcal{O}}(\cdot)$ contains constant factors that depend on both the local and global geometry of the best-response-region $\mathcal{R}_{j^{*}}$. For a fully formal statement that includes these factors, see Lemma 6.)

The full proof of Theorem 2, deferred to Appendix A.2, involves some technical steps to achieve as good as possible a scaling in $N$. The caveat of Theorem 2 is that commitment power can be robustly exploited in this way only if there are enough observations of the commitment. One obvious requirement is that the best-response-region $\mathcal{R}_{j^{*}}$ needs to be non-empty in $\mathbb{R}^{m-1}$. Second, the number of observations $N$ needs to be greater than the effective dimension of the game for the leader, $m$. This is a natural requirement to ensure that the follower has learned at least a meaningful estimate of the commitment. Third, the “constant” factors in Theorem 2 actually reflect properties about both the local and global geometry of the polytope; see Appendix A.2 for more details. Intuitively, geometric properties that lead to undesirable scaling in the constant factors in the robustness guarantee are listed below:

The Stackelberg commitment being a “pointy” vertex: this can lead to a commitment being far away from the boundary in certain directions, but closer in others, making it more likely for a different response to be elicited. 2. 2.

Local constraints being very different from global constraints, which implies that commitments too far in the interior of the local feasibility set will no longer satisfy all the constraints of the best-response-region.

Even with these caveats, Theorem 2 provides an attractive general framework for constructing robust commitments by making a natural connection to interior-point methods in optimization121212Noting that interior point methods are provably polynomial-time algorithms to solve LPs, it is plausible to think that in fact, stopping the interior point method appropriately early would also give us a robustness guarantee - which would imply that finding optimal robust commitments is even easier than finding optimal commitments!. We observe significant empirical benefit from the constructions in the simulations in Section 4.

We also mention a couple of special cases of leader-follower games for which the robust commitment constructions of Theorem 2 are not required; in fact, it is simply optimal to play Stackelberg.

Remark 1**.**

For games in which the mixed-strategy Stackelberg equilibrium coincides with a pure strategy, the follower’s best response is always as expected regardless of the number of observations. There is no tradeoff and it is simply optimal to play Stackelberg even under observational uncertainty.

Remark 2**.**

For the zero-sum case, it was observed [BHP14] that a Stackelberg commitment is made assuming that the follower will respond in the worst case. If there is observational uncertainty, the follower can only respond in a way that yields payoff for the leader that is better than expected. This results in an expected payoff greater than or equal to the Stackelberg payoff $f^{*}_{\infty}$, and it simply makes sense to stick with the Stackelberg commitment $\mathbf{x}^{*}_{\infty}$. As we have seen, this logic does not hold up for non-zero-sum games because different responses can lead to worse-than-expected payoff. One way of thinking of this is that the function $f_{\infty}(.)$ can generally be discontinuous in $\mathbf{x}$ for a non-zero-sum game, but is always continuous for the special case of zero-sum.

3.3 Approximating the maximum possible payoff

So far, we have considered the limited-observability problem and shown that the Stackelberg commitment $\mathbf{x}^{*}_{\infty}$ is not a suitable choice. We have constructed robust commitments that come close to idealized Stackelberg payoff $f^{*}_{\infty}$ and shown that the guarantee fundamentally depends on the number of observations scaling with the effective dimension of the game. Now, we turn to the question of whether we can approximate $f^{*}_{N}$, the actual optimum of the program. Note that since the problem is in general non-convex in $\mathbf{x}$, it is NP-hard to exactly compute.

Rather than the traditional approach of constructing a polynomial-time-approximation-algorithm, our approach is approximation-theoretic131313In other words, the extent of approximation is measured by the number of samples as opposed to the runtime of an algorithm. This is very much the flavor of previously-obtained results on Stackelberg zero-sum security games [BHP14].. We first show that in the large-sample case, we cannot do much better than the actual Stackelberg payoff $f^{*}_{\infty}$; informally speaking, our ability to fool the follower into responding strictly-better-than-expected is limited. Combining this with the robust commitment construction of Theorem 2, we obtain an approximation to the optimum payoff.

The main result of this section is stated below.

Theorem 3**.**

We have

[TABLE]

for some constant $C>0$ depending on the parameters of the game $(A,B)$.

As a corollary the commitment construction defined in Theorem 2 provides a $\widetilde{\mathcal{O}}(\sqrt{\frac{1}{N}})$-additive-approximation algorithm to $f^{*}_{N}$. The proof of Theorem 3 is provided in the appendix.

Intellectually, Theorem 3 tells us that the robust commitments are essentially optimal. The practical benefit that Theorem 3 affords us is that we now have an approximation to the optimum payoff the leader could possibly obtain, which can be computed in constant time after computing the Stackelberg equilibrium, which itself is polynomial time [CS06]. This is because the robust commitment is obtained by first computing Stackelberg equilibrium $\mathbf{x}^{*}_{\infty}$, and then deviating away from $\mathbf{x}^{*}_{\infty}$ in the magnitude and direction specified. We will now study the empirical benefits of our robust commitment constructions.

4 Simulations

4.1 Example $2\times 2$ and $2\times 3$ games

First, we return to the non-zero-sum games described in Examples 2 and  3. These were $2\times 2$ and $2\times 3$ games respectively, and the Stackelberg commitment was non-robust for both games. Now, armed with the results in Theorem 2, we can employ our robust commitment constructions and study their performance. To construct our robust commitments, we first computed the Stackelberg commitment using the LP solver in scipy (scipy.optimize.linprog), and then used the construction in Theorem 2.

Figures 3(a) and  4(b) compares the expected payoff obtained by our robust commitment construction scheme $\{\mathbf{x}_{N}\}_{N\geq 1}$ for different numbers of samples $N$, and for the games described in Examples 2 and  3 respectively. The benchmark with respect to which we measure this expected payoff is the Stackelberg payoff $f^{*}_{\infty}$ (obtained by Stackelberg commitment under infinite observability and tie-breakability in favor of the leader). We also observe a significant gap between the payoffs obtained by these robust commitment constructions and the payoff obtained if we used the Stackelberg commitment $\mathbf{x}^{*}_{\infty}$. We showed in theory that there is significant benefit for choosing the commitment to factor in such observational uncertainty, and we can now see it in practice.

Furthermore, for the case of $2$ leader actions we were able to brute-force the maximum possible obtainable payoff141414First we used scipy.optimize.brute with an appropriate grid size to initialize, and then ran a gradient descent at that initialization point. This was feasible for the case of $2$ pure strategies. $f^{*}_{N}$, and compare the value to the robust commitment payoff. This comparison is particularly valuable for smaller values of $N$, as shown in Figures 3(b) and 4(c). We notice that the values are much closer even than our theory would have predicted, and even for small values of $N$. Thus, our constructions have significant practical benefit as well: we are able to get close to the optimum while drastically reducing the required computation (to just solving $n$ LPs!).

Since these examples involved $2\times n$ games, the commitment construction became trivial (i.e. only one direction to move along) – next, we test our commitment constructions for $m\times m$ security games. Instead of looking at specific examples, we now look at a random ensemble to see what behavior ensues.

4.2 Random security games

Our next set of simulations is inspired by the security games framework. We create a random ensemble of $5\times 5$ security games in which the defender can defend one of $5$ targets, and the attacker can attack one of these $5$ targets. The defender and attacker rewards are chosen to be uniformly at random between $[0,1]$, and their penalties are uniform at random between $[-1,0]$. This is essentially the random ensemble that was created in previous empirical work on security games [AKK*+*12]. Figure 5 shows the construction of this ensemble.

The purpose of random security games is to show that the properties we observed above – unstable Stackelberg commitment, robust commitment payoff approximating the optimum – are the norm rather than the exception. Figure 6 illustrates the results for random security games. The performance of the sequence of robust commitments $\{\mathbf{x}_{N}\}_{N\geq 1}$, as well as the Stackelberg commitment $\mathbf{x}^{*}_{\infty}$ is plotted in Figure 6(a) against the benchmark of idealized Stackelberg performance $f^{*}_{\infty}$. Figure 6(b) depicts the rate of convergence of the gap in robust commitment performance to the idealized Stackelberg payoff – we can clearly see the $\mathcal{O}(\frac{1}{\sqrt{N}})$ rate of convergence in this plot. Finally, Figure 6(c) plots the percentage gap between robust commitment payoff and idealized Stackelberg payoff as a function of $N$.

We can make the following conclusions from these plots:

The Stackelberg commitment is extremely non-robust on average. In fact we noticed that this was the case with high probability. This happens because the Stackelberg commitment, although it can vary widely for different games in the random ensemble, is very likely on a boundary shared with other responses and therefore unstable. 2. 2.

The robust commitments are doing much better on average than the original Stackelberg commitment even for very large values of $N$. The stark difference in payoff between the two motivates the construction of the robust commitment, which was as easy to compute as the Stackelberg commitment.

5 Proof sketches

In this section we describe briefly the philosophy for the proofs of our main theorems. To understand the strong lack of robustness in Stackelberg equilibrium, it is essential to visualize the best-response-regions of the leader, i.e. subsets of the mixed strategy space for which the follower best response is a particular pure strategy. (Note that there are $n$ such best-response regions.) Figure 7 depicts an illustration of these best-response-regions, with the region corresponding to the follower’s best response to the Stackelberg commitment highlighted in red. The figure shows the Stackelberg commitment at a vertex (extreme-point) of the best-response polytope $\mathcal{R}_{j^{*}}$; this is generally the case [CS06].

First, the reason for the strong instability of Stackelberg commitment to even an infinitesmal amount of uncertainty can be seen from Figure 7: an infinitesmal amount of fluctuation in how the leader commitment is observed will make the follower respond with a different pure strategy with constant probability, corresponding to the regions depicted in purple. Because of the tie-breaking assumption, it turns out that the expected payoff from any of these alternate responses is strictly worse than the Stackelberg payoff. These facts are proved formally using the Berry-Esseen theorem. Note that an uncertainty in commitment could also lead to a response from one of the yellow regions in the figure (which could either hurt or benefit the leader), but the probability of this happening turns out to decay exponentially.

This observation implied that the optimality of the Stackelberg commitment under ideal assumptions was exactly what made it suboptimal under a small amount of uncertainty; we exploit this to construct the robust commitment constructions of Theorem 2. The qualitative idea is to push the commitment to a small extent into the interior of the best-response-region $\mathcal{R}_{j^{*}}$ so that it simultaneously satisfies a property of being “close” to the Stackelberg commitment, while also ensuring that the fluctuations in its empirical estimate are highly likely to stay in $\mathcal{R}_{j^{*}}$ (which guarantees that the identity of the best response of the follower is preserved). For the special case of $m=2$, this is a simple tradeoff to navigate as there is only one direction in which one can move into the interior. For higher dimensions, we take inspiration from the rich literature on interior-point methods and, in fact, use Dikin ellipsoids [KN12] for both the commitment construction and analysis. Ensuring that the fluctuations of the commitment preserve the follower best response with high probability, in particular, requires sophisticated tail bounds on discrete distribution learning and a careful consideration of the best-response-polytope geometry.

The proof of Theorem 3 ties several facts that we have seen formally, as well as alluded to, together. First, a generalization of Theorem 1 tells us that we cannot improve sizeably over Stackelberg by committing to any mixed strategy on the boundary between two or more best-response regions. Second, we show that the improvement gained by a fixed commitment in the interior of any best-response-region decreases exponentially with $N$, simply because the probability of eliciting a better-than-expected response decreases exponentially with $N$. Putting these two facts together, the natural thing to try would be commitments that approach a boundary as $N$ increases (much like our robust commitment constructions, but now with a different motive). This should happen fast enough that we maintain a sizeable probability of eliciting a different response for every $N$, while simultaneously ensuring that that different response is actually better-than-expected. We then show that the ensuing gain over and above Stackelberg would have to decrease with $N$ according to the rate specified.

6 Conclusions and Discussion

We constructed robust commitment constructions with several advantages. First, we are able to effectively preserve the Stackelberg payoff by ensuring a high-probability guarantee on the follower responding as expected. An oblique, but significant philosophical advantage to our robust commitments is that their guarantees hold even if we removed the pivotal assumption of follower breaking ties in favor of the leader. We essentially showed that as the number of observations $N$ grows, our construction naturally converges to the Stackelberg commitment at a specific rate. We also motivated that the constructions, which were inspired by interior point geometry, are computable in polynomial time given the Stackelberg commitment.

Second, we established fundamental limits on the ability of the leader to gain over and above Stackelberg payoff. We formally showed that this ability disappears in the large-sample regime, and in a certain sense that our robust commitments are approximately optimal. Our results established a formal connection between leader payoff and follower discrete distribution learning, and in the context of these limits, both players are mutually incentivized to increase learnability under limited samples, even though the setting is non-cooperative – which was a rather surprising conclusion.

Our work provides implications for both leader and follower payoffs when the leader is known to be committed to a fixed strategy, but the commitment can only be revealed partially. However, our model took commitment establishment for granted, i.e. the follower assumed that the leader would indeed be drawing its pure strategies iid from the same mixture in every round. The partial reputation setting should most generally be modeled as a repeated game (either with a finite-horizon or discounted model), in which the belief in commitment needs to be built up over time. Studying the problem of finite observability of commitment in isolation is, in our view, an important first step towards eventually solving this problem, which poses many modeling challenges in itself. In earlier rounds, directly responding to the empirical estimate of leader commitment will be suboptimal for the follower. Instead, he may want to maintain a possibility that the leader will play minimax/Nash and respond accordingly. From the point of view of the leader, if the iid assumption is removed, an interesting question is whether the leader could choose to play more deterministically, in such a way to increase strategy learnability while maintaining a follower impression of iid commitment. By doing this, the leader could establish commitment faster but also run the risk of looking too deterministic/predictable in time, in which case the follower would take undue advantage. Conversely, the leader may not even be incentivized to increase follower learnability in the finitely repeated, or discounted setting.

Finally, it is interesting to think about the applicability of the robust commitment perspective to algorithmically more difficult problems like Bayesian persuasion and public/private signalling games with multiple followers, which can have observational limitations on information transfer in much the same way as has been described for the applications in this paper.

Acknowledgments

We thank the anonymous reviewers for valuable feedback. We gratefully acknowledge the support of the NSF through grant AST-1444078, and the Berkeley ML4Wireless research center.

Appendix A Proofs

Before moving into the proofs themselves, we define some additional notation.

Definition 3**.**

The set of alternate follower best response to the mixed commitment $\mathbf{x}$ is denoted by

[TABLE]

We will be particularly interested in this set for the Stackelberg commitment, that is, $\mathcal{K}^{*}_{\mathsf{alt}}(\mathbf{x}^{*}_{\infty})$. In general, the set will be non-empty as the follower could be agnostic between more than one pure strategy in response – it is only responding with the pure strategy $j^{*}$ to break ties in the leader’s favor. Figure 7 shows this demarcation of follower responses into the expected response $j^{*}$, and alternate responses to the Stackelberg commitment $\mathbf{x}^{*}_{\infty}$.

Further, we denote maximum and minimum obtainable leader payoffs respectively by

[TABLE]

A.1 Proof of Theorem 1

We consider a general $2\times n$ game and denote the Stackelberg probability of leader playing pure strategy $1$ by $p^{*}_{\infty}$. Recall that $p^{*}_{\infty}\in(0,1)$ (since we have assumed for the proof that the Stackelberg commitment is mixed). Let $j_{\mathsf{alt}}$ be the alternate response to the Stackelberg commitment, i.e. we have $\mathcal{K}^{*}(p^{*}_{\infty})=\{j^{*}_{\infty},j_{\mathsf{alt}}\}$. Without loss of generality, the best-response regions can be described as

[TABLE]

Finally, we define $f^{(2)}:=\lim_{\epsilon\to 0}f_{\infty}(p^{*}_{\infty}+\epsilon)$. Since we are considering leader-follower games for which the function $f_{\infty}(.)$ is discontinuous at $p^{*}_{\infty}$, by the tie-breaking assumption on Stackelberg commitment we will have $f^{(2)}<f^{*}_{\infty}$.

Now, we consider the quantity $f_{N}(p^{*}_{\infty})$. Denoting $\widehat{P}_{N}$ as the empirical estimate of the quantity $p^{*}$, we have

[TABLE]

We will now proceed to bound the probabilities $T_{1}(N)$ and $T_{2}(N)$.

First, we deal with the quantity $T_{2}(N)$, which reflects the probability of a mismatched response that is neither Stackelberg nor the alternate response on the boundary. By the Hoeffding bound, we have

[TABLE]

Denoting $C^{\prime\prime}:=2\left(\min\{p^{+}-p^{*}_{\infty},p^{*}_{\infty}-p^{-}\}\right)^{2}$, we then have

[TABLE]

and as expected, this probability decays exponentially with $N$.

Next, we deal with the quantity $T_{1}(N)$, which reflects the probability of eliciting the alternate response on the Stackelberg boundary. We show that this event is non-vanishingly probable.

We define the following quantities

[TABLE]

Recall that $Z_{N}$ is a real-valued random variable. We denote its cumulative distribution function by $F_{N}(.)$.

By a simple change of variables, we then have

[TABLE]

Now, recall that $S_{N}=\sum_{j=1}^{N}I_{j}$ for iid random variables $I_{j}\sim\mbox{Ber}(p^{*}_{\infty})$. Also note that since we have considered games with mixed Stackelberg commitment, we have $0<p^{*}_{\infty}<1$. We now invoke the first half of the classical Berry-Esseen theorem [Ber41, Ess42] stated here as a lemma.

Lemma 1**.**

There exists a positive constant $C$ such that if $I_{1},I_{2},\ldots$ are iid random variables with $\mathbb{E}[I_{1}]=\mu<\infty$, $\operatorname{var}(I_{1})=\sigma^{2}>0$ and $\mathbb{E}[|I_{1}-\mu|^{3}]=\rho<\infty$, we have

[TABLE]

for all $x\in\mathbb{R}$, where $\phi(.)$ denotes the CDF of the standard normal distribution $\mathcal{N}(0,1)$.

It is easy to verify that the distribution $I_{1}\sim\mbox{Ber}(p^{*}_{\infty})$ satisfies the above conditions. Therefore, we can directly apply Lemma 1 and get

[TABLE]

for positive constant $C>0$, thus giving

[TABLE]

Substituting for the expressions for $T_{1}(N)$ and $T_{2}(N)$, we now have

[TABLE]

which corresponds exactly to Equation (2). Clearly, the right hand side of this equation is decreasing in $N$ and so the first corollary – that $f_{N}(p^{*}_{\infty})\leq f^{*}_{\infty}$ for $N\geq N_{0}$ – holds. Precisely, we have

[TABLE]

and so we have

[TABLE]

This is the second corollary from Theorem 1 and completes the proof. ∎

A.2 Proof of Theorem 2

A.2.1 Notation

For this proof, it will be convenient to consider the $(m-1)$-dimensional representation of the probability simplex, i.e.

[TABLE]

Then, we can represent a commitment $\mathbf{x}\in\Delta_{m}$ by its $(m-1)$-dimensional representation $\mathbf{y}=\begin{bmatrix}x_{1}&x_{2}&\ldots&x_{m-1}\end{bmatrix}$, and the leader payoff if the follower were to respond with pure strategy $j\in[n]$ by

[TABLE]

where we have

[TABLE]

Similarly, we can represent the corresponding follower payoff by

[TABLE]

where we have

[TABLE]

We can also represent this representation of the empirical estimate of $\mathbf{y}$ from $N$ samples by $\mathbf{\widehat{Y}}_{N}$, and this representation Stackelberg commitment by $\mathbf{y}^{*}_{\infty}$.

Now, we can consider all the functions introduced in Section 2.2 in terms of the commitment $\mathbf{x}$ and equivalently define them in terms of the $(m-1)$-dimensional representation of the commitment, $\mathbf{y}$.

We also denote the $p$th operator norm of a matrix by $|\!|\!|.|\!|\!|_{{p}}$.

A.2.2 The commitment construction

We consider the $(m-1)$-dimensional representation of the best-response-region corresponding to the Stackelberg commitment, $\mathcal{R}_{j^{*}}$. There are many things to consider while constructing a robust commitment. The first, and obvious, one would be that the follower should respond the same way as it would to Stackelberg when it observes the full mixture. That is, we would have $j^{*}(\mathbf{y}_{N})=j^{*}$ or alternatively stated, $\mathbf{y}_{N}\in\mathcal{R}_{j^{*}}$.

Intuitively, the expected payoff of a leader commitment under observational uncertainty, particularly in terms of gap to the optimal Stackelberg payoff, will depend on two factors: one, how likely the follower is to respond the same as it would if it observed the full commitment; and two, how “far” the leader commitment mixture is from the optimal Stackelberg commitment mixture. We qualitatively show this dependence in the following lemma.

Lemma 2**.**

Consider a commitment $\mathbf{y}_{N}$ for which we can provide the following guarantee:

[TABLE]

We then have

[TABLE]

Proof.

We have

[TABLE]

Recall that we have $f^{*}_{\infty}=\langle\mathbf{y}^{*}_{\infty},\,\mathbf{c}_{j^{*}}\rangle+d_{j^{*}}$. Therefore, the gap from Stackelberg is bounded as

[TABLE]

where the second inequality follows from Holder’s inequality. This proves the lemma. ∎

This lemma implies that we want a commitment construction $\mathbf{y}_{N}$ with the following two-fold guarantee151515Interestingly, the fact that $\mathbf{y}^{*}_{\infty}$ is on an extreme point of $\mathcal{R}_{j^{*}}$ will imply that the two conditions are at odds with one another, and we will need to trade them off. For instance, choosing $\mathbf{y}_{N}=\mathbf{y}^{*}_{\infty}$ would satisfy the second condition perfectly by being as close as possible to the Stackelberg commitment, but there would be no guarantee on the best-response as it lies on the boundary of the best-response region..

$\|\mathbf{y}_{N}-\mathbf{y}^{*}_{\infty}\|_{1}$ is bounded (and ideally vanishes with $N$). 2. 2.

$\mathbf{\widehat{Y}}_{N}\in\mathcal{R}_{j^{*}}$ with high probability.

A.2.3 Commitment construction using localized geometry

We will leverage the special structure of the Dikin ellipsoid [KN12] used in interior-point methods to make our commitment constructions. Observe that $\mathbf{y}^{*}_{\infty}$ is always going to be on an extreme point (vertex) of the best-response-polytope161616Recall that the Stackelberg equilibrium is the solution to the LP defined on the best-response-polytope [CS06]. $\mathcal{R}_{j^{*}}$. We now collect the $k=|\mathcal{K}^{*}(\mathbf{x}^{*}_{\infty})|$ constraints that are satisfied with equality at $\mathbf{x}^{*}_{\infty}$:

[TABLE]

This is simply the constraint set for commitments such that the follower prefers to respond with pure strategy $j^{*}$ over any pure strategy $j\in\mathcal{K}^{*}(\mathbf{y}^{*}_{\infty})$ (i.e. any pure strategy whose corresponding best-response-polytope shares a boundary with the Stackelberg best-response-polytope at point $\mathbf{y}^{*}$), and can be thought of as the set of local constraints to the Stackelberg vertex in the best-response polytope $\mathcal{R}_{j^{*}}$. We also collect the other constraints that describe $\mathcal{R}_{j^{*}}$:

[TABLE]

and together with the local constraints at the Stackelberg vertex, these describe the global constraints for the polytope.

We represent the system of inequalities in matrix form as: $B\mathbf{y}\preceq\mathbf{c}$ for some $B\in\mathbb{R}^{k\times(m-1)}$ and some $\mathbf{c}\in\mathbb{R}^{k}$. We leverage the following useful fact about a general set of linear constraints.

Fact 1**.**

For any parameterization of linear constraints $(B,\mathbf{c})$, there exists an affine transformation $\mathbf{y}^{\prime}=T_{1}\mathbf{y}+T_{2}$ (where $T_{1}\in\mathbb{R}^{(m-1)\times(m-1)}$ is invertible and $T_{2}\in\mathbb{R}^{m-1}$) and a matrix $B^{\prime}\in\mathbb{R}^{k\times(m-1)}$ such that

[TABLE]

We denote the transformation function by $T(\cdot)$ and its inverse by $T^{-1}(\cdot)$. In particular, we note the relationship $B=B^{\prime}T_{1}$.

The above fact is useful171717A subtle point is that there do exist special cases of polytope constraints for which Fact 1 is true only with an augmentation of the variable space from $m$ to $2m$ dimensions. Then, defining the invertible map becomes trickier. Nevertheless, for ease of exposition and clarity in the proof, we assume that we can indeed carry out the affine transformation without augmenting the dimension. because it is most convenient to define our class of commitments in the transformed space $\mathbf{y}^{\prime}=T(\mathbf{y})$.

Definition 4**.**

For a particular value of $\delta\in(0,1)$, Stackelberg commitment $\mathbf{y}^{*}_{\infty}$, and local constraints modeled by $(B,\mathbf{c})$, we define a $\delta$-deviation commitment by

[TABLE]

Our robust commitments $\{\mathbf{y}_{N}\}_{N\geq 1}$ are going to be taken out of the set of $\delta$-deviation commitments, with appropriately chosen values of $\{\delta_{N}\}_{N\geq 1}$. Clearly, the computational complexity of constructing any $\delta$-deviation commitment is equivalent to the complexity of computing the Stackelberg equilibrium itself.

To understand how to set these values, we will turn to the question of how to satisfy the three conditions above.

First, we observe that $\mathbf{y}(\delta;\mathbf{y}^{*}_{\infty})$ satisfies the local constraints $B\mathbf{y}\preceq\mathbf{c}$ for any $\delta\in(0,1)$. Because of Fact 1, it suffices to show that its affine transformation $\mathbf{y}^{\prime}(\delta;(\mathbf{y}^{*})^{\prime}_{\infty})$ satisfies the local constraints $B^{\prime}\mathbf{y}^{\prime}\preceq\mathbf{1}$. Recall that $(\mathbf{y}^{*})^{\prime}_{\infty}$ satisfies all the local constraints with equality, i.e. we have $B^{\prime}(\mathbf{y}^{*})^{\prime}_{\infty}=\mathbf{1}$. From the definition of the commitment, we thus have

[TABLE]

Next, we turn to the question of how close such a defined commitment would be from the Stackelberg commitment $\mathbf{y}^{*}_{\infty}$, in terms of the $\ell_{1}$ norm. For this, we have

[TABLE]

Therefore, we have

[TABLE]

In lieu of Lemma 2, we wish to choose values $\{\delta_{N}\}_{N\geq 1}$ (to create commitments $\{\mathbf{y}_{N}\}_{N\geq 1}$) such that $\delta_{N}$ decreases with $N$ sufficiently fast, while maintaining a high probability of staying in the best-response polytope $\mathcal{R}_{j^{*}}$. To understand the rate at which we can decrease $\delta_{N}$, we need to prove a high-probability best-response guarantee.

A.2.4 Using the local Dikin ellipsoid as a confidence ball

For a (affine-transformed) commitment $\mathbf{y}^{\prime}(\delta;(\mathbf{y}^{*})^{\prime}_{\infty})$, we make use of the local Dikin ellipsoid centered at $\mathbf{y}^{\prime}(\delta;(\mathbf{y}^{*})^{\prime}_{\infty})$, defined below for an arbitrary point $\mathbf{y}^{\prime}$.

Definition 5** ( [KN12]).**

For constraint set $B^{\prime}\mathbf{y}^{\prime}\preceq\mathbf{1}$, the Dikin ellipsoid of radius $r$ centered at $\mathbf{y}^{\prime}$ is given by

[TABLE]

where we define

[TABLE]

The Dikin ellipsoid has two special properties [KN12]:

Affine invariance*: (using the notation from Fact 1) For transformation $\mathbf{y}^{\prime}=T(\mathbf{y})$, the Dikin ellipsoid of radius $r$ centered at the point $\mathbf{y}$ for the polytope $B\mathbf{y}\preceq\mathbf{c}$ is $\mathbb{B}_{B,\mathbf{c},\mathbf{y}^{\prime}}(r)=T^{-1}(\mathbb{B}_{B^{\prime},\mathbf{1},\mathbf{y}^{\prime}}(r))$.* 2. 2.

Interior guarantee*: For any interior point $\mathbf{y}^{\prime}$ (according to the constraint set $B^{\prime}\mathbf{y}^{\prime}\preceq\mathbf{1}$), the Dikin ellipsoid of radius $1$ centered at $\mathbf{y}^{\prime}$ is contained in the feasibility set, that is,*

[TABLE]

We center our Dikin ellipsoid at $\mathbf{y}^{\prime}(\delta;(\mathbf{y}^{*})^{\prime}_{\infty})$, and observe that the constraint takes on a particularly nice form, as stated by the following simple lemma.

Lemma 3**.**

For any $\delta\in(0,1)$, the Dikin ellipsoid can be expressed as

[TABLE]

Furthermore, in the original space we can write

[TABLE]

Proof.

From Definition 4, we observe that $B^{\prime}\mathbf{y}^{\prime}(\delta;(\mathbf{y}^{*})^{\prime}_{\infty})=(1-\delta)B^{\prime}(\mathbf{y}^{*})^{\prime}_{\infty}=(1-\delta)\mathbf{1}$. This implies that

[TABLE]

and thus we have

[TABLE]

where in the last equality step, we have used $(B^{\prime})^{\top}B^{\prime}=\sum_{i=1}^{k}(\mathbf{b}^{\prime})_{i}(\mathbf{b}^{\prime})_{i}^{\top}$, noting that $(\mathbf{b}^{\prime})_{i}$ denotes the $i^{th}$ row of $B^{\prime}$.

Thus, the ellipsoid constraint in Equation (9) can be rewritten as

[TABLE]

thus completing the first part of the proof (Equation (11)).

For the second part of the proof, we use the affine invariance property of the Dikin ellipsoid, which tells us that

[TABLE]

Now, observe that

[TABLE]

where in the last step we have used the relationship $B=B^{\prime}T_{1}$ from Fact 1. Putting these observations together, we have

[TABLE]

completing the second part of the proof. ∎

At this stage, it is worth remembering that the commitment is mixed, and the payoff from using a $\delta$-deviation commitment $\mathbf{y}(\delta;\mathbf{y}^{*}_{\infty})\in\Delta_{m-1}$ under a finite number of observations $N$ depends on the guarantee that its observed empirical distribution $\mathbf{\widehat{Y}}_{N}$ (typically) stays inside the best-response region. As a starting point we need to guarantee that at least the local vertex constraints are not violated.

Note that $\mathbf{y}(\delta;\mathbf{y}^{*}_{\infty})\in\Delta_{m-1}$ is an interior point for any $\delta>0$, and thus the interior guarantee property of the Dikin ellipsoid can be applied. We thus know that if the empirical distribution of the commitment stays inside the Dikin ellipsoid centered at the actual commitment, it will stay inside the local constraint feasibility set. Thus, it makes sense to use the Dikin ellipsoid as a confidence ball and tail bound the probability that the empirical estimate lies outside this ball. Because of the weighted $\ell_{2}$-ball structure on the particular ellipsoid corresponding to a $\delta$-deviation commitment that we proved in Lemma 3, this is not difficult to do. We state this formally in the following lemma.

Lemma 4**.**

For a given $\delta>0$, let $\mathbf{\widehat{Y}}_{N}$ be the empirical distribution of $N$ samples drawn from the $\delta$-deviation commitment $\mathbf{y}(\delta;\mathbf{y}^{*}_{\infty})$. Then, we have

[TABLE]

provided that $N\geq\frac{20m|\!|\!|B|\!|\!|_{{\mathsf{op}}}^{2}}{\delta^{2}}$.

Proof.

The proof is a simple consequence of Devroye’s lemma [Dev83], which tail bounds the total variation between the empirical estimate of a discrete distribution and the true distribution.

Lemma 5** ([Dev83]).**

Let $\mathbf{\widehat{Y}}_{N}$ be the empirical distribution of $N$ samples drawn from any distribution $\mathbf{y}\in\Delta_{m-1}$. Then, as long as $\delta\geq\sqrt{\frac{20m}{N}}$ we have

[TABLE]

We note from Lemma 3 that

[TABLE]

and thus, we have

[TABLE]

where inequality $\mathsf{(i)}$ uses the definition of the operator norm and inequality $\mathsf{(ii)}$ uses the fact that $\|\mathbf{v}\|_{2}\leq\|\mathbf{v}\|_{1}$ for any finite-dimensional vector $\mathbf{v}$. Applying Lemma 5 directly then gives us

[TABLE]

as long as

[TABLE]

This completes the proof.

∎

A.2.5 Completing proof of Theorem 2: Ensuring global constraint satisfiability

Let us recap what we have proved so far about a $\delta$-deviation commitment $\mathbf{y}(\delta;\mathbf{y}^{*}_{\infty})$ for any $\delta\in(0,1)$.

For $N$ samples from $\mathbf{y}(\delta;\mathbf{y}^{*}_{\infty})$, we have $\Pr\left[\mathbf{\widehat{Y}}_{N}\notin\mathbb{B}_{B,\mathbf{c},\mathbf{y}(\delta;\mathbf{y}^{*}_{\infty})}(1)\right]\leq 3\exp\{-\frac{N\delta^{2}}{25|\!|\!|B|\!|\!|_{{\mathsf{op}}}^{2}}\}$ (from Lemma 4). 2. 2.

$\|\mathbf{y}(\delta;\mathbf{y}^{*}_{\infty})-\mathbf{y}^{*}_{\infty}\|_{1}\leq\delta$.

Thus, from Lemma 2 we have for any $\delta$-deviation commitment,

[TABLE]

Thus, if we had $\mathbb{B}_{B,\mathbf{c},\mathbf{y}(\delta;\mathbf{y}^{*}_{\infty})}(1)\subset\mathcal{R}_{j^{*}}$, we would have

[TABLE]

However, the set $\mathcal{R}_{j^{*}}$ includes global constraints in addition to the local constraints $B\mathbf{y}\preceq\mathbf{c}$, and all points in the local Dikin ellipsoid need not satisfy these constraints. This is the final technicality in the proof that we now deal with. We will see that for a small enough value of $\delta$ (that depends on how the local geometry of the polytope relates to the global geometry), we can guarantee global satisfiability. Let the constraints corresponding to the convex polytope $\mathcal{R}_{j^{*}}$ be represented by $C\mathbf{y}\preceq\mathbf{d}$, and the corresponding constraints after the affine transformation $(T_{1},T_{2})$ be represented as $C^{\prime}\mathbf{y}^{\prime}\preceq\mathbf{d}^{\prime}$ (where the values of $\mathbf{d}^{\prime}$ corresponding the local constraints are $1$). Thus, for the vertex $(\mathbf{y}^{*})^{\prime}_{\infty}$, we can define the quantity

[TABLE]

Because $\mathcal{R}_{j^{*}}$ is non-empty and convex, we have $\mathcal{Z}(\mathcal{R}_{j^{*}};(\mathbf{y}^{*})^{\prime}_{\infty})>0$.

From this definition, under the condition $\delta<\mathcal{Z}(\mathcal{R}_{j^{*}};(\mathbf{y}^{*})^{\prime}_{\infty})$ we have

[TABLE]

where the last implication is because of the affine-invariance property of the Dikin ellipsoid.

On the other hand, we used the condition $N\geq\frac{20m|\!|\!|B|\!|\!|_{{\mathsf{op}}}^{2}}{\delta^{2}}$ to prove Lemma 4. Combining these inequalities tells us that we require $N>\frac{20m|\!|\!|B|\!|\!|_{{\mathsf{op}}}^{2}}{\mathcal{Z}(\mathcal{R}_{j^{*}};(\mathbf{y}^{*})^{\prime}_{\infty})^{2}}=\mathcal{O}(m)$ to prove our result.

Then, we formally define our robust commitment for a particular value of $N$ below, and prove this final lemma which is essentially a formal statement of Theorem 2.

Lemma 6**.**

For every $N>\frac{20m|\!|\!|B|\!|\!|_{{\mathsf{op}}}^{2}}{\mathcal{Z}(\mathcal{R}_{j^{*}};(\mathbf{y}^{*})^{\prime}_{\infty})^{2}}$, and every $p<1/2$, we define the $p$-robust commitment as a $\delta_{N,p}$-deviation commitment $\mathbf{y}_{N,p}:=\mathbf{y}(\delta_{N,p};\mathbf{y}^{*}_{\infty})$, where

[TABLE]

We then have

[TABLE]

Proof.

This is a simple consequence of everything put together. Since $N>m$, we have $\delta_{N,p}<\mathcal{Z}(\mathcal{R}_{j^{*}};(\mathbf{y}^{*})^{\prime}_{\infty})$ and thus we have $\mathbb{B}_{B,\mathbf{1},\mathbf{y}(\delta_{N,p};\mathbf{y}^{*}_{\infty}}(1)\subset\mathcal{R}_{j^{*}}$. This tells us that

[TABLE]

and thus from Lemma 2 we get the following expression:

[TABLE]

Directly substituting the expression for $\delta_{N,p}$ in Equation (13) into the above expression completes the proof. ∎

A.3 Proof of Theorem 3

Recall that $f^{*}_{N}:=\max_{\mathbf{x}\in\Delta_{m}}f_{N}(\mathbf{x})$. To prove an upper bound on $f^{*}_{N}$, we will upper bound $f_{N}(\mathbf{x})$ for every $\mathbf{x}\in\Delta_{m}$.

Without loss of generality the same proof method will extend to all $\mathbf{x}\in\Delta_{m}$. Denoting as shorthand $p_{j}(\mathbf{x}):=\Pr\left[\mathbf{\widehat{X}}_{N}\in\mathcal{R}_{j}\right]$, we have

[TABLE]

where we denote $T_{j}(\mathbf{x}):=p_{j}(\mathbf{x})\langle\mathbf{a}_{j},\,\mathbf{x}\rangle$. We will proceed to upper bound the quantity $T_{j}(\mathbf{x})$ for every $\mathbf{x}\in\Delta_{m}$ and every $j\in[n]$.

To do this, we will see that it is natural to divide all the pure strategy responses that are possible to a commitment $\mathbf{x}$ into three categories. The first is the expected response $j^{*}(\mathbf{x})$. The second is the set of responses whose regions are adjacent to the expected response as defined below.

Definition 6**.**

For a particular commitment $\mathbf{x}\in\Delta_{m}$, the set of adjacent-to-expected responses $\mathcal{K}^{*}_{aug}(\mathbf{x})$ is the set of all best-responses whose corresponding best-response-regions share a boundary with the best-response-region corresponding to the best response to $\mathbf{x}$. Formally, we have

[TABLE]

We also define $\mathcal{K}_{\mathsf{far}}:=[n]-(\{j^{*}(\mathbf{x})\}\cup\mathcal{K}^{*}_{\mathsf{aug}}(\mathbf{x}))$ as the set of all follower responses that are “far” from the expected response in this sense.

The illustration in Figure 8 shows this division.

For the rest of the proof, we will drop the term $\mathbf{x}$ from the notation and denote $\mathcal{K}^{*}_{\mathsf{aug}}:=\mathcal{K}^{*}_{\mathsf{aug}}(\mathbf{x})$ as well as $j^{*}:=j^{*}(\mathbf{x})$. This is done for notational simplicity.

It is first easy to show a bound on $T_{j^{*}}(\mathbf{x})$. In particular, we can directly use the definition of the function $f_{\infty}(.)$ to obtain

[TABLE]

This inequality is also intuitive because the leader would only hope to gain from eliciting a different-than-expected response. Next, we deal with this cases.

A.3.1 “Far”-from-expected responses

We collect the set of commitments that (if observed fully) would elicit a response far away from the actual expected response. Formally, we denote $\mathcal{R}_{\mathsf{far}}:=\cup_{j\in\mathcal{K}_{\mathsf{far}}}\mathcal{R}_{j}$. Now, we wish to bound the term

[TABLE]

By definition, we have $\text{cl}(\mathcal{R}_{j^{*}})\cap\text{cl}(\mathcal{R}_{far})=\emptyset$. Because we are considering finite games, i.e. $n<\infty$, there exists a constant $C>0$ that depends solely on the parameters of the game such that

[TABLE]

Geometrically, Figure LABEL:fig:thm3illustration shows this separation between the expected-response-region and any far-from-expected-response-region. To understand the probability of eliciting such responses, we invoke a classical result from large-deviations theory, Sanov’s theorem [CK11]. The upper bound part of the theorem is restated here as a lemma and with appropriate notation.

Lemma 7**.**

Let $I_{1},I_{2},\ldots,I_{N}$ be $\text{i.i.d }\sim\mathbf{x}$ for any $\mathbf{x}\in\Delta_{m}$ and $\mathbf{\widehat{X}}_{N}$ denote the empirical estimate. Then, for any region $\mathcal{R}\subseteq\Delta_{m}$, we have

[TABLE]

Combining equations (20) and (19), we therefore get

[TABLE]

The rationale for calling these responses far-from-expected is now clear: there is a minimum constant separation in terms of the KL-divergence from the expected best response, and so the probability of realizing these responses decreases exponentially with $N$.

Dealing with the adjacent-to-expected responses is more delicate. We turn to this case next.

A.3.2 Adjacent-to-expected responses

Consider the set of adjacent-to-expected response $\mathcal{K}^{*}_{\mathsf{aug}}$. We wish to bound the term $\sum_{j\in\mathcal{K}^{*}_{\mathsf{aug}}}T_{j}(\mathbf{x})$. It turns out that we can no longer control the probability that one of these responses is elicited for all choices of $\mathbf{x}\in\mathcal{R}_{j^{*}}$ – this is because the commitment $\mathbf{x}$ could be chosen arbitrarily close to a boundary of its expected-response-region. However, we can bound the ensuing payoff as a function of how close the commitment is to a boundary. This notion of closeness is defined in terms of the $\ell_{1}$-norm below.

Definition 7**.**

For a commitment $\mathbf{x}\in\mathcal{R}_{j^{*}}$ and a particular adjacent response $j\in\mathcal{K}^{*}_{\mathsf{aug}}$, we define its minimum distance to the boundary by

[TABLE]

First, we use this notion to bound the maximum possible payoff that could be elicited.

Lemma 8**.**

For any commitment $\mathbf{x}\in\mathcal{R}_{j^{*}}$, we have

[TABLE]

Proof.

Let $\widetilde{\mathbf{x}}\in{\arg\min}_{\mathbf{x}^{\prime}\in\text{cl}(\mathcal{R}_{j})}\|\mathbf{x}-\mathbf{x}^{\prime}\|_{1}$. (Note that the minimum exists because we’ve taken the closure of the region.) Using Holder’s inequality, we have

[TABLE]

For every $j\in\mathcal{K}^{*}_{aug}$ we have

[TABLE]

where we are crucially using the fact that $\widetilde{\mathbf{x}}$ lies on the boundary and the tie-breaking assumption, to tie its payoff to the function $f_{\infty}(.)$. Substituting the above bound into the definition of $T_{j}(\mathbf{x})$ completes the proof. ∎

Lemma 8 is important because it limits the potential of leader gain from eliciting an adjacent follower response, even if she is able to do this with high probability, i.e. by committing very close to a boundary. Figure 9 clearly illustrates this for a $2\times 2$ game: here, the leader might wish to elicit different-than-expected response $2$ with high probability. However, to do this she would have to commit close to the boundary between regions expecting responses $1$ and $2$, resulting in her payoff being close to an objective function value of $f_{\infty}(.)$ (in the figure, depicted as the optimum payoff $f^{*}_{\infty}$). For a general $m\times n$ game, the picture stays the same.

Since the quantity $\delta_{1}(\mathbf{x})$ can take values anywhere in the interval $[0,2]$ (by the triangle inequality), we will still want to control the quantity $p_{j}(\mathbf{x})$ for large enough values of $\delta$. We will again use Devroye’s lemma (Lemma 5) for tail bounding the total variation between the empirical estimate of a distribution and a true distribution. Recall that the condition required for it to be applied was $\delta\geq\sqrt{\frac{20m}{N}}$.

It is natural to further divide the set $\mathcal{K}^{*}_{\mathsf{aug}}$ into two subsets, defined by the commitment $\mathbf{x}$.

[TABLE]

Let’s consider these subsets one-by-one. First, we use Lemma 8 and the definition of the subset $\mathcal{K}^{*}_{\mathsf{aug},1}(\mathbf{x})$ to get

[TABLE]

Next, we consider the term $\sum_{j\in\mathcal{K}^{*}_{\mathsf{aug},2}(\mathbf{x})}T_{j}(\mathbf{x})$. We state and prove the following lemma.

Lemma 9**.**

For any commitment $\mathbf{x}\in\Delta_{m}$, we have

[TABLE]

Proof.

Consider any $j\in\mathcal{K}^{*}_{\mathsf{aug},2}(\mathbf{x})$. Now note that by the definition of $\delta_{1}(\mathbf{x};j)$, we can denote the open $\ell_{1}$ ball with center $\mathbf{x}$ and radius $\delta_{1}(\mathbf{x})$ by $B_{1}(\mathbf{x};\delta_{1}(\mathbf{x}))$. By the definition of $\delta_{1}(\mathbf{x};j)$, it follows that $B_{1}(\mathbf{x};\delta_{1}(\mathbf{x};j))\cap\mathcal{R}_{j}=\emptyset$. Therefore, we have

[TABLE]

where we used Lemma 5 in the last inequality since we have $\mathcal{K}^{*}_{\mathsf{aug},2}(\mathbf{x})$, we have $\delta_{1}(\mathbf{x})\geq\sqrt{\frac{20m}{N}}$.

Combining this with Lemma 8, we then have

[TABLE]

Next, it is easy to verify that the function $g_{2}(\delta)=\delta\exp\{-\frac{N\delta^{2}}{25}\}$ is decreasing in $\delta$ over the domain $\delta\geq\sqrt{\frac{20m}{N}}$ for all $m\geq 1$. This tells us that

[TABLE]

and so we have

[TABLE]

Summing over all $j\in\mathcal{K}^{*}_{\mathsf{aug},2}(\mathbf{x})$ and substituting Equation (27) then proves the lemma.

∎

A.3.3 Putting it all together

Combining Equations (16), (21), (A.3.2) and (26) into Equation (14), we have

[TABLE]

for some constant $C>0$. This inequality holds for any $\mathbf{x}\in\Delta_{m}$. This implies that $f^{*}_{N}\leq f^{*}_{\infty}+Cn\sqrt{\frac{m}{N}}$, thus completing the proof of Theorem 3. ∎