Regression Equilibrium

TL;DR

This paper introduces a game-theoretic framework for strategic prediction algorithms, analyzing their equilibrium behavior and demonstrating convergence and learnability properties within a PAC learning setting.

Contribution

It develops a novel game-theoretic model for strategic prediction, proving equilibrium existence, convergence of learning dynamics, and sample efficiency for equilibrium approximation.

Findings

Algorithms may mispredict intentionally for strategic advantage.

The empirical game always has a pure Nash equilibrium.

Learning dynamics converge to an approximate equilibrium with few samples.

Abstract

Prediction is a well-studied machine learning task, and prediction algorithms are core ingredients in online products and services. Despite their centrality in the competition between online companies who offer prediction-based products, the \textit{strategic} use of prediction algorithms remains unexplored. The goal of this paper is to examine strategic use of prediction algorithms. We introduce a novel game-theoretic setting that is based on the PAC learning framework, where each player (aka a prediction algorithm aimed at competition) seeks to maximize the sum of points for which it produces an accurate prediction and the others do not. We show that algorithms aiming at generalization may wittingly mispredict some points to perform better than others on expectation. We analyze the empirical game, i.e., the game induced on a given sample, prove that it always possesses a pure Nash equilibrium, and show that every better-response learning process converges. Moreover, our learning-theoretic analysis suggests that players can, with high probability, learn an approximate pure Nash equilibrium for the whole population using a small number of samples.

Figures

Equations

$$\text{Pdim}(H)=\max \left\{m\in \mathbb{N}:\exists \mathcal{S}\in {\mathcal{X}}^m\text{ such that }\mathcal{S}\text{ is pseudo-shattered by }H\right\}.$$

$$\mathcal{I}(z,g)=\{\begin{array}{ll}1& \left|g(x)-y\right|\le t\\ 0& \text{otherwise}\end{array}.$$

$$w_i(z;\mathbf{h})=\{\begin{array}{ll}0& \text{ if }\mathcal{I}(z,h_i)=0\\ \frac{1}{{\sum }_{i^{\prime }=1}^N\mathcal{I}(z,h_{i^{\prime }})}& \text{otherwise}\end{array}.$$

$${\pi }_i(\mathbf{h})=E_{z\sim \mathcal{D}}\left[w_i(z;\mathbf{h})\right].$$

$${\pi }_i^{\mathcal{S}}(\mathbf{h})=E_{z\in \mathcal{S}}\left[w_i(z;\mathbf{h})\right]=\frac{1}{m}\sum _{j=1}^m{w}_i(z_j;\mathbf{h}).$$

$$\mathcal{W}\stackrel{\text{def}}{=}=\{w(z;\mathbf{h})\mid \mathbf{h}\in \mathcal{H}\}.$$

$${\mathcal{F}}_i\stackrel{\text{def}}{=}\left\{\mathcal{I}(z,h)\mid h\in {\mathcal{H}}_i\right\}.$$

$$\mathcal{F}\stackrel{\text{def}}{=}\prod _{i=1}^N{\mathcal{F}}_i=\left\{\mathcal{I}\left(z,\mathbf{h}\right)\mid \mathbf{h}\in \mathcal{H}\right\}.$$

$$\mathcal{F}\cap \mathcal{S}=\prod _{i=1}^N({\mathcal{F}}_i\cap \mathcal{S})=\left\{\left(\mathcal{I}(z_1,\mathbf{h}),\dots ,\mathcal{I}(z_m,\mathbf{h})\right)\mid \mathbf{h}\in \mathcal{H}\right\}.$$

$$\left|\mathcal{F}\cap \mathcal{S}\right|=\prod _{i=1}^N\left|{\mathcal{F}}_i\cap \mathcal{S}\right|.$$

$$\underset{\mathcal{S}\sim {\mathcal{D}}^m}{\mathrm{Pr}}\left(\exists \mathbf{h}:\left|{\pi }_i(\mathbf{h})-{\pi }_i^{\mathcal{S}}(\mathbf{h})\right|\ge ϵ\right)\le 4{\Pi }_{\mathcal{W}}(2m)e^{-\frac{{ϵ}^{2}m}{8}}.$$

$$\underset{\mathcal{S}\sim {\mathcal{D}}^m}{\mathrm{Pr}}\left(\exists i\in \mathcal{N}:\underset{\mathbf{h}\in \mathcal{H}}{\sup }\left|{\pi }_i(\mathbf{h})-{\pi }_i^{\mathcal{S}}(\mathbf{h})\right|\ge ϵ\right)\le 4N(2em{)}^{10{\sum }_{i=1}^N{d}_i}{e}^{-\frac{{ϵ}^{2}m}{8}}.$$

$$\Phi (h_i^{\prime },{\mathbf{h}}_{-i})-\Phi (h_i,{\mathbf{h}}_{-i})={\pi }_i(h_i^{\prime },{\mathbf{h}}_{-i})-{\pi }_i(h_i,{\mathbf{h}}_{-i}).$$

$$m\ge \frac{320d}{{ϵ}^2}\log \left(\frac{160d}{{ϵ}^2}\right)+\frac{160d\log (2e)}{{ϵ}^2}+\frac{16}{{ϵ}^2}\log \left(\frac{4N}{\delta }\right).$$

$$\forall i\in \mathcal{N}:\underset{\mathbf{h}\in \mathcal{H}}{\sup }\left|{\pi }_i(\mathbf{h})-{\pi }_i^{\mathcal{S}}(\mathbf{h})\right|<ϵ.$$

$$M(h_i)=\left(w_i(z_1,{\mathbf{h}}_{-i},h_i),\dots ,w_i(z_m,{\mathbf{h}}_{-i},h_i)\right).$$

$$y=\{\begin{array}{ll}{a}_{1}x+b_1+ϵ& \text{w.p. }\frac{1}{2}\\ a_{2}x+b_2+ϵ& \text{w.p. }\frac{1}{2}\end{array},$$

$$y=\{\begin{array}{ll}{a}_{1}x+b_1+ϵ& \text{if }x\le x_0\\ a_{2}x+b_2+ϵ& \text{else}\end{array},$$

$$\mathcal{D}(x,y,t)=\{\begin{array}{ll}\frac{1}{2}& 0\le x<1,y=0,t=\frac{1}{2}\\ \frac{1}{2}& 1\le x\le 2,y=1,t=\frac{1}{2}\\ 0& \text{otherwise}\end{array}.$$

$$h^{\mathcal{S}\to 0}(x)=\{\begin{array}{ll}0& \exists y,t:(x,y,t)\in \mathcal{S}\\ \text{mathds}{1}_{1\le x\le 2}& \forall y,t:(x,y,t)\notin \mathcal{S}\end{array},h^{\mathcal{S}\to 1}(x)=\{\begin{array}{ll}1& \exists y,t:(x,y,t)\in \mathcal{S}\\ \text{mathds}{1}_{1\le x\le 2}& \forall y,t:(x,y,t)\notin \mathcal{S}\end{array}.$$

$${\mathcal{H}}_1=\{h^{\mathcal{S}\to 0}\mid \mathcal{S}\subset \mathcal{Z}\}\cup \{h^{\mathcal{S}\to 1}\mid \mathcal{S}\subset \mathcal{Z}\}\cup \{h^0,h^1\}.$$

$$B(\tilde{z};\mathbf{h})=\left\{i\in \mathcal{N}:\left|h_i(x)-y\right|\le \underset{i^{\prime }\in \mathcal{N}}{\min }\left|h_{i^{\prime }}(x)-y\right|\right\}.$$

$${\tilde{w}}_i(\tilde{z};\mathbf{h})=\{\begin{array}{ll}\frac{1}{|B(\tilde{z};\mathbf{h})|}& i\in B(\tilde{z};\mathbf{h})\\ 0& \text{otherwise}\end{array}.$$

$${\tilde{\pi }}_i(\mathbf{h})=E_{\tilde{z}\sim \tilde{\mathcal{D}}}\left[{\tilde{w}}_i(\tilde{z};\mathbf{h})\right],{\tilde{\pi }}_i^{\mathcal{S}}(\mathbf{h})=E_{z\in \tilde{\mathcal{S}}}\left[{\tilde{w}}_i(\tilde{z};\mathbf{h})\right].$$

$$U\stackrel{\text{def}}{=}\left([0,1]\times [1,2]\right)\cup \left([1,2]\times [0,1]\right)\cup \left([2,3]\times [1,2]\right).$$

$${\mathcal{G}}^{\ge }=\{g_h^{\ge }(x,r)=\text{mathds}{1}_{h(x)\ge r}\mid h\in {\mathcal{H}}_i,(x,r)\in \mathcal{X}\times \mathbb{R}\},$$

$${\mathcal{G}}^{\le }=\{g_h^{\le }(x,r)=\text{mathds}{1}_{h(x)\le r}\mid h\in {\mathcal{H}}_i,(x,r)\in \mathcal{X}\times \mathbb{R}\}.$$

$$2^{5d_i}$$

$$>\sum _{j=0}^{d_i}{\left(\frac{10e{d}_i}{j}\right)}^j\stackrel{\text{Claim refclaim:binomialfactors}}{\ge }\sum _{j=0}^{d_i}\left(\genfrac{}{}{0ex}{}{10d_i}{j}\right)$$

$$\ge {\Pi }_{{\mathcal{G}}^{\ge }}(10d_i)={\Pi }_{{\mathcal{G}}^{\le }}(10d_i);$$

$${\Pi }_{{\mathcal{F}}_i}(10d_i)\le {\Pi }_{{\mathcal{G}}^{\ge }}(10d_i){\Pi }_{{\mathcal{G}}^{\le }}(10d_i)<2^{10d_i}.$$

$${\mathcal{S}}^{\prime }=\left((x_1,r_1),\dots ,(x_m,r_m)\right),$$

Full text

1 Introduction

Prediction plays an important role in twenty-first century economics. In a prediction task, an algorithm is given a sequence of examples composed of labeled instances, and its goal is to learn a general rule that maps instances to labels. With the recent data explosion, commercial companies can, like never before, collect massive amounts of data and employ sophisticated machine learning algorithms to discover patterns and seek connections between different observations. For instance, after examining a sufficient number of apartments – their characteristics and selling prices – real estate experts may attempt to accurately predict the selling price of a new, unseen apartment. Typically, the quality of a prediction algorithm is measured by its success in predicting the value of an unlabeled (or unseen) instance.

However, ubiquitously prediction is not carried out in isolation. For revenue-seeking companies, prediction is another tool that can be exploited to increase revenue. To illustrate, consider several competing real estate experts, who provide prediction service for the selling value of apartments on their websites. These experts gain directly from user traffic to their websites, and hence aim to attract as many users as possible. A user, after receiving the experts’ predictions and selling his111For ease of exposition, third-person singular pronouns are “he” for a user and “she” for a player. apartment, can evaluate which experts were accurate222Crucially, we assume that the predicted values and the actual ones are independent. This is arguably the case if experts are external, and not buyers nor real-estate brokers. Consequently, users care for accurate prediction and not over-estimation. and which were not; he will typically decide that an expert is accurate based on his experience and/or the experience of his family and friends. That user may decide to have future interaction with one/some of the accurate experts, or recommend his friends to interact with them; thus, providing an accurate prediction to a user translates to higher revenue. Interestingly, maximizing revenue and minimizing discrepancy do not coincide in this example, as further illustrated in Figure 1, suggesting that prediction algorithms in competition should optimize revenue explicitly, and not other measures that affect revenue only indirectly. Despite the intuitive clarity of this tradeoff and the enormous amount of work done on prediction in the machine learning and statistical learning communities, far too little attention has been paid to the study of prediction in the context of competition.

In this paper we study how the selection of prediction algorithms is affected by strategic behavior in a competitive setting, using a game-theoretic lens. We consider a space of users, where each user is modeled as a triplet $(x,y,t)$ of an instance, a label and a threshold, respectively. An instance encodes public information describing the user, e.g., a real vector with his apartment’s characteristics; the label is the value that should be predicted, e.g., the actual selling value of his apartment (that is only revealed later); and the threshold is the “distance” he is willing to accept between a proposed prediction and his label. Namely, a prediction $f(x)$ is said to be accurate w.r.t. the user associated with $(x,y,t)$ if $\left|f(x)-y\right|$ is less than or equal to $t$. In such a case, the user is satisfied and willing to adopt the prediction, granting one monetary unit to the expert who produced the accurate prediction. If a user is satisfied with several predictions (of several experts), he selects one uniformly at random. Indeed, the user-model we adopt is aligned with the celebrated “Satisficing" principle of Simon [26], and other widely-accepted models in the literature on choice prediction, e.g., the model of selection based on small samples [5, 11]. Several players (companies or experts) are equipped with infinite strategy spaces, or hypothesis classes in learning-theoretic terminology. A player’s strategy space models the possible predictive functions she can employ. Players are competing for the users, and a player’s payoff is the expected number of users who find her predictions to be accurate, in the aforementioned sense. To model uncertainty w.r.t. the labels and thresholds, we use the PAC-learning framework of Valiant [27]. We assume user distribution is unknown, but the players have access to a sequence of examples, containing instances, labels and thresholds, with which they should optimize their payoffs w.r.t. the unknown underlying user distribution.

From a machine learning perspective, we now face the challenge of what would be a good prediction algorithm profile, i.e., a set of algorithms for the players such that no player would deviate from her algorithm assuming the others all stick to their algorithms. Indeed, such a profile of algorithms determines a pure Nash equilibrium (PNE), a powerful solution concept which rarely exists in games. An important question in this regard is whether such a profile exists. An accompanying question is whether learning dynamics in which players may change their prediction algorithms to better respond to others would converge. Therefore, we ask:

• •

Does a PNE exist?

• •

Will the players be able to find it efficiently with high probability using better-response dynamics?

We prove that the answer to both questions is in the affirmative. We first show that when the capacity of each strategy space is bounded (i.e., finite pseudo-dimension), players can learn payoffs from samples. Namely, we show that the payoff function of each player uniformly converges over all possible strategy profiles (that include strategies of the other players); thus, with high probability a player’s payoff under any strategy profile is not too distant from her empirical payoff. Later, we show that an empirical PNE always exists, i.e., a PNE of the game induced by the empirical sample distribution. Moreover, we show that any learning dynamics in which players improve their payoff by more than a non-negligible quantity converges fast to an approximate PNE. Using the two latter results, we show an interesting property of the setting: the elementary idea of sampling and better-responding according to the empirical distribution until convergence leads to an approximate PNE of the game on the whole population. We analyze this learning process, and formalize the above intuition via an algorithm that runs in polynomial time in the instance parameters, and returns an approximate PNE with high probability.

To further exemplify the setting, we then consider the case of players employing linear regression. We modify the algorithm of Ben-Porat and Tennenholtz [6], proposed for a related setting, to devise an algorithm for the best linear response of a player. We conduct simulations on various synthetic datasets to visualize the structure of PNEs in several settings, and to perform sensitivity analysis. Finally, we discuss two extensions. We consider the case of infinite capacities, and demonstrate that non-learnability can occur even if the user distribution is known to all players. We also study a natural variant of the model, where each user grants one monetary unit to the most accurate player, and show that learning and PNE existence are no longer guaranteed.

1.1 Related work

The intersection of research in game theory and machine learning has increased rapidly in recent years. Sample empowered mechanism design is a fruitful line of research. For example, Cole and Roughgarden [9], Gonczarowski and Nisan [13], Morgenstern and Roughgarden [21] reconsider auctions where the auctioneer can sample from bidder valuation functions, thereby relaxing the assumption of prior knowledge on bidder valuation distribution [22]. Empirical distributions also play a key role in other lines of research [2, 4, 17], where e.g. Babichenko et al. [4] show how to obtain an approximate equilibrium by sampling any mixed equilibrium. The PAC-learning framework proposed by Valiant [27] has also been extended by Blum et al. [7], who consider a collaborative game where players attempt to learn the same underlying prediction function, but each player has her own distribution over the space. In their work each player can sample from her own distribution, and the goal is to use information sharing among the players to reduce the sample complexity. Another line of work deals with extracting data from selfish agents for a variety of machine learning tasks [8, 10, 19, 14]. We stress that these works are only parallel to this one, as in this paper there are no strategic agents, only myopic users who follow the “best” prediction they get. Moreover, in this work the strategic parties are those who provide the predictions, aiming at satisfying as many users as possible.

Our work is inspired by Dueling Algorithms [16]. Immorlica et al. [16] analyze an optimization problem from the perspective of competition, rather than from the point of view of a single optimizer. That work examines the dueling form of several optimization problems, e.g., the shortest path from the source vertex to the target vertex in a graph with random weights. While minimizing the expected length is a probable solution concept for a single optimizer (i.e., socially optimal), this is no longer the case in the defined duel. Our model is also related to Competing Bandits [18]. Mansour et al. [18] consider a competition between two bandit algorithms faced with the same sample, where users arrive one by one and choose between the two algorithms. This work is substantially different than Mansour et al. [18]. To name a few differences, our work players also share the same sample but we consider an offline setting and not an online one; infinite strategy spaces and not a finite set of actions; context in the form of property vector for each user; and an arbitrary number of asymmetric players, where asymmetry is reflected in the strategy space of each player.

Most relevant to our work is the work by Ben-Porat and Tennenholtz [6]. The authors present a learning task where a newcomer agent is given a sequence of examples, and wishes to learn a best-response to the players already on the market. They assume that the agent can sample triplets composed of instance, label and current market prediction, and define the agent’s payoff as the proportion of points (associated with users) she predicts better than the other players. Indeed, Ben-Porat and Tennenholtz [6] introduce a learning task incorporating economic interpretation into the objective function of the (single) optimizer, but in fact does not provide any game-theoretic analysis. In contrast, this paper considers game-theoretic interaction between players, and its main contribution lies in the analysis of such interactions. Since learning dynamics consists of steps of unilateral deviations that improve the deviating player’s payoff, the Best Response Regression of Ben-Porat and Tennenholtz [6] can be thought of as an initial step to this work.

1.2 Our contribution

Our contribution is three-fold. First, we explicitly suggest that prediction algorithms, like other products on the market, are in competition. This novel view emphasizes the need for stability in prediction-based competition similar to Hotelling’s stability in spatial competition [15].

Second, we introduce an extension of the PAC-learning framework for dealing with strategy profiles, each of which is a sequence of functions. To illustrate why this is not an immediate application of the PAC-learning framework, recall that in the PAC-learning framework we assume a loss function which quantifies how good or bad a hypothesis is, regardless of the hypothesis space. In our games, however, a strategy profile cannot be determined to be good (being a PNE) or bad (there exist beneficial deviations) by itself; we must as well consider all profiles and in particular the possible beneficial deviations.

We show a reduction from payoff maximization to a single-player loss minimization setting, which is later used to achieve bounds on the sample complexity for uniform convergence over the set of profiles. We also show that when players have approximate better-response oracles, they can learn an approximate PNE of the empirical game. The main technical contribution of this paper is an algorithm which, given $\epsilon,\delta$, samples a polynomial number of points in the game instance parameters, runs any $\epsilon$-better-response dynamics, and returns an $\epsilon$-PNE with probability of at least $1-\delta$.

Third, we consider games with at least one player with an infinite pseudo-dimension. We show a game instance where each player can learn the best prediction function from her hypothesis class if she were alone in the game, but a PNE of the empirical game is not generalized. This inability to learn emphasizes that strategic behavior can introduce further challenges to the machine learning community.

2 Problem definition

In this section we formalize the model. We begin with an informal introduction to elementary concepts in both game theory and learning theory that are used throughout the paper.

Game theory

A non-cooperative game is composed of a set of players $\mathcal{N}=\{1,\dots,N\}$; a strategy space $\mathcal{H}_{i}$ for every player $i$; and a payoff function $\pi_{i}:\mathcal{H}_{1}\times\cdots\times\mathcal{H}_{N}\rightarrow\mathbb{R}$ for every player $i$. The set $\mathcal{H}=\mathcal{H}_{1}\times\cdots\times\mathcal{H}_{N}$ contains all possible strategy profiles, and a tuple of strategies $\boldsymbol{h}=(h_{1},\dots h_{N})\in\mathcal{H}$ is called a strategy profile, or simply a profile. We denote by $\boldsymbol{h}_{-i}$ the vector obtained by omitting the $i$-th component of $\boldsymbol{h}$.

A strategy $h_{i}^{\prime}\in\mathcal{H}_{i}$ is called a better response of player $i$ with respect to a strategy profile $\boldsymbol{h}$ if $\pi_{i}(h_{i}^{\prime},\boldsymbol{h}_{-i})>\pi_{i}(\boldsymbol{h})$. Similarly, $h_{i}^{\prime}$ is said to be an $\epsilon$-better response of player $i$, w.r.t. a strategy profile $\boldsymbol{h}$ if $\pi_{i}(h_{i}^{\prime},\boldsymbol{h}_{-i})\geq\pi_{i}(\boldsymbol{h})+\epsilon$, and a best response to $\boldsymbol{h}_{-i}$ if $\pi_{i}(h_{i}^{\prime},\boldsymbol{h}_{-i})\geq\sup_{h_{i}\in\mathcal{H}_{i}}\pi_{i}(h_{i},\boldsymbol{h}_{-i})$ .

We say that a strategy profile $\boldsymbol{h}$ is a pure Nash equilibrium (herein denoted PNE) if every player plays a best response under $\boldsymbol{h}$. We say that a strategy profile $\boldsymbol{h}$ is an $\epsilon$-PNE if no player has an $\epsilon$-better response under $\boldsymbol{h}$, i.e., for every player $i$, it holds that $\pi_{i}(\boldsymbol{h})\geq\sup_{h_{i}^{\prime}\in\mathcal{H}_{i}}\pi_{i}(h_{i}^{\prime},\boldsymbol{h}_{-i})-\epsilon$.

Learning theory

Let $\mathcal{X}$ be a set, and let $F$ be a class of binary-valued functions $F\subseteq{\{0,1\}}^{\mathcal{X}}$. Given a sequence $\mathcal{S}=(x_{1},\dots x_{m})\in\mathcal{X}^{m}$, we denote the restriction of $F$ to $\mathcal{S}$ by $F\cap\mathcal{S}=\left\{\left(f(x_{1}),\dots,f(x_{m})\right)\mid f\in F\right\}$. The growth function of $F$, denoted $\Pi_{F}:\mathbb{N}\rightarrow\mathbb{N}$, is defined as $\Pi_{F}(m)=\max_{\mathcal{S}\in\mathcal{X}^{m}}\left|F\cap\mathcal{S}\right|$. We say that $F$ shatters $\mathcal{S}$ if $\left|F\cap\mathcal{S}\right|=2^{\left|\mathcal{S}\right|}$. The Vapnik-Chervonenkis dimension of a binary function class is the cardinality of the largest set of points in $\mathcal{X}$ that can be shattered by $F$, $\textnormal{VCdim}(F)=\max\left\{m\in\mathbb{N}:\Pi_{F}(m)=2^{m}\right\}$.

Let $H$ be a class of real-valued functions $H\subseteq\mathbb{R}^{\mathcal{X}}$. The restriction of $H$ to $\mathcal{S}\in\mathcal{X}^{m}$ is analogously defined, $H\cap\mathcal{S}=\left\{\left(h(x_{1}),\dots,h(x_{m})\right)\mid h\in H\right\}$. We say that $H$ pseudo-shatters $\mathcal{S}$ if there exists $\boldsymbol{r}=(r_{1},\dots,r_{m})\in\mathbb{R}^{m}$ such that for every binary vector $\boldsymbol{b}=(b_{1},\dots b_{m})\in\{-1,1\}^{m}$ there exists $h_{\boldsymbol{b}}\in H$ and for every $i\in[m]$ it holds that $\operatorname{sign}(h_{\boldsymbol{b}}(x_{i})-r_{i})=b_{i}$. The pseudo-dimension of $H$ is the cardinality of the largest set of points in $\mathcal{X}$ that can be pseudo-shattered by $H$,

[TABLE]

2.1 Model

We consider a set of users who are interested in a prediction provided by a set of competing players. Each user is associated with a vector $(x,y,t)$, where $x$ is the instance; $y$ is the label; and $t$ is the threshold that the user is willing to accept.

The players offer predictions to the users. When a user associated with a vector $(x,y,t)$ approaches player $i$, she produces a prediction $h_{i}(x)$. If $\left|h_{i}(x)-y\right|$ is at most $t$, the user associated with $(x,y,t)$ will grant one monetary unit to player $i$. Otherwise, that user will move on to another player. We assume that users approach players according to the uniform distribution, although our model and results support any distribution over player orderings. Player $i$ has a set of possible strategies (prediction algorithms) $\mathcal{H}_{i}$, from which she has to decide which one to use. Each player aims to maximize her expected payoff, and will act strategically to do so.

Formally, the game is a tuple $\langle\mathcal{Z},\mathcal{D},\mathcal{N},\mathcal{H},\pi\rangle$ such that

$\mathcal{Z}$ is the examples domain $\mathcal{Z}=\mathcal{X}\times\mathcal{Y}\times\mathcal{T}$, where $\mathcal{X}\subset\mathbb{R}^{n}$ is the instance domain; $\mathcal{Y}\subset\mathbb{R}$ is the label domain; and $\mathcal{T}\subset\mathbb{R}_{\geq 0}$ is the tolerance domain. 2. 2.

$\mathcal{D}$ is a probability distribution over $\mathcal{Z}=\mathcal{X}\times\mathcal{Y}\times\mathcal{T}$. 3. 3.

$\mathcal{N}$ is the set of players, where $\mathcal{N}=[N]\stackrel{{\scriptstyle\text{def}}}{{=}}\{1,\dots,N\}$. A strategy of player $i$ is an element from $\mathcal{H}_{i}\subseteq\mathcal{Y}^{\mathcal{X}}$. The space of all strategy profiles is denoted by $\mathcal{H}=\times_{i=1}^{N}\mathcal{H}_{i}$. 4. 4.

For $z=(x,y,t)$ and a function $g:\mathcal{X}\rightarrow\mathcal{Y}$, we define the indicator $\mathcal{I}(z,g)$ to be 1 if the distance between the value $g$ predicted for $x$ and $y$ is at most $t$. Formally,

[TABLE] 5. 5.

Given a strategy profile $\boldsymbol{h}=(h_{1},\dots h_{N})$ with $h_{i}\in\mathcal{H}_{i}$ for $i\in\mathcal{N}$ and $z=(x,y,t)\in\mathcal{Z}$, let

[TABLE]

Note that $w_{i}(z;\boldsymbol{h})$ represents the expected payoff of player $i$, w.r.t. the user associated with $z$. The payoff of player $i$ under $\boldsymbol{h}$ is the average sum over all users, and is defined by

[TABLE] 6. 6.

$\mathcal{D}$ is unknown to the players.

We assume players have access to a sequence of examples $\mathcal{S}$, drawn i.i.d. from $\mathcal{D}$. Given a game instance $\langle\mathcal{Z},\mathcal{D},\mathcal{N},\mathcal{H},\pi\rangle$ and a sample $\mathcal{S}=\{z_{1},\dots z_{m}\}$, we denote by $\langle\mathcal{Z},\mathcal{S}\sim\mathcal{D}^{m},\mathcal{N},\mathcal{H},\pi\rangle$ the empirical game: the game over the same $\mathcal{N},\mathcal{H},\mathcal{Z}$ and uniform distribution over the known $\mathcal{S}\in\mathcal{Z}^{m}$. We denote the payoff of player $i$ in the empirical game by

[TABLE]

When $\mathcal{S}$ is known from the context, we occasionally use the term empirical PNE to denote a PNE of the empirical game. Since the empirical game is a complete information game, players can use the sample in order to optimize their payoffs.

The optimization problem of finding a best response in our model is intriguing in its own right. Throughout the paper, we assume that each player $i$ has a polynomial $\epsilon$-better-response oracle. Namely, given a real number $\epsilon>0$, a strategy profile $\boldsymbol{h}$ and sample $\mathcal{S}$, we assume that each player $i$ has an oracle that returns an $\epsilon$-better response to $\boldsymbol{h}_{-i}$ if such exists or answers false otherwise, which runs in time $\text{poly}(\frac{1}{\epsilon},m,N)$. Despite that this assumption may be non-trivial in general cases, we show in Section 4 such a best response oracle for the case of $\mathcal{H}_{i}$ being the class of linear functions with constant input dimension (denoted by $n$ in the model above). We also discuss situations where a better response cannot be computed efficiently in Section 6, and present the applicability of our model for these cases as well.

3 Meta algorithm and analysis

Throughout this section we assume that the pseudo-dimension of $\mathcal{H}_{i}$ is finite, and we denote it by $d_{i}$, i.e., $\textnormal{Pdim}(\mathcal{H}_{i})=d_{i}<\infty$. Our goal is to propose a generic method for finding an $\epsilon$-PNE efficiently. The method is composed of two steps: first, it attains a sample of “sufficient” size. Afterwards, it runs an $\epsilon$-better-response dynamics until convergence, and returns the obtained profile. The underlying idea is straightforward, but its analysis is non-trivial. In particular, we need to show two main claims:

• •

Given a sufficiently large sample $\mathcal{S}$, the payoff of each player $i$ in the empirical game is not too far away from her payoff in the actual game, with high probability. This holds concurrently for all possible strategy profiles.

• •

An $\epsilon$-PNE exists in every empirical game. Therefore, players can reach an $\epsilon$-PNE of the empirical game quickly, using their $\epsilon$-better-response oracles.

These claims will be made explicit in forthcoming Subsections 3.1 and 3.2. We formalize the above discussion via Algorithm 1 in Subsection 3.3.

3.1 Uniform convergence in probability

We now bound the probability (over all choices of $\mathcal{S}$) of having player $i$’s payoff (for an arbitrary $i\in\mathcal{N}$) greater or less than its empirical counterpart by more than $\epsilon$. Notice that the restriction of $\mathcal{H}_{i}$ to any arbitrary sample $\mathcal{S}$, i.e., $\mathcal{H}_{i}\cap\mathcal{S}$, may be of infinite size. Nevertheless, the payoff under a strategy profile $\boldsymbol{h}\in\mathcal{H}$ concerns the functions $w_{1},\dots,w_{N}$ only and not the actual real-valued predictions produced by the players under $\boldsymbol{h}$; hence, uniform convergence of another set of functions should be argued, the class of functions that compose the payoff with the predictions.

More formally, given $z\in\mathcal{Z}$ and $\boldsymbol{h}\in\mathcal{H}$, let $w(z;\boldsymbol{h})=(w_{1}(z;\boldsymbol{h}),\dots,w_{N}(z;\boldsymbol{h}))$, and denote by $\mathcal{W}$ the class of functions from $\mathcal{Z}$ to $\{1,\frac{1}{2},\dots,\frac{1}{N},0\}^{N}$ such that

[TABLE]

Notice that the set $\mathcal{W}\cap\mathcal{S}$ is finite. Moreover, for some $z\in\mathcal{Z}$, $w(z;\boldsymbol{h})=w(z;\boldsymbol{h}^{\prime})$ may occur for $\boldsymbol{h}\neq\boldsymbol{h}^{\prime}$; thus, to argue that uniform convergence holds, one would typically show that the growth function of $\mathcal{W}$ is polynomial. However, analyzing $\mathcal{W}$ directly seems tricky. To circumvent a direct analysis of $\mathcal{W}$, we define auxiliary classes of binary functions $(\mathcal{F}_{i})_{i\in\mathcal{N}}$. Let $\mathcal{F}_{i}$ be a class of functions from $\mathcal{Z}$ to $\{0,1\}$ such that

[TABLE]

In the rest of this subsection, we prove properties of $\mathcal{F}_{i}$ and then associate those properties with the payoffs of the game.

Observe that $\mathcal{F}_{i}$ is a binary function class, and thus its complexity can be quantified using the VC dimension. On the other hand, we already know that the pseudo-dimension of $\mathcal{H}_{i}$, which is a real-valued function class, is $d_{i}$. The following combinatorial lemma bounds the VC dimension of $\mathcal{F}_{i}$ as a function of the pseudo-dimension of $\mathcal{H}_{i}$. By doing so, we present an interesting connection between these two celebrated notations of expressive power of a space of functions (the VC dimension and the pseudo-dimension).

Lemma 3.1**.**

$\textnormal{VCdim}(\mathcal{F}_{i})\leq 10d_{i}$.

After discovering the connection between the growth rate of $\mathcal{H}_{i}$ and $\mathcal{F}_{i}$, we can progress to bounding the growth of the product function class $\mathcal{F}$ (which we will define shortly). For ease of notation, denote $\mathcal{I}(z,\boldsymbol{h})=(\mathcal{I}(z,h_{1}),\dots,\mathcal{I}(z,h_{N}))$. Let $\mathcal{F}$ be a class of functions from $\mathcal{Z}$ to $\{0,1\}^{N}$ defined by

[TABLE]

Note that every element in $\mathcal{F}$ is a function from $\mathcal{Z}$ to $\{0,1\}^{N}$. The restriction of $\mathcal{F}$ to a sample $\mathcal{S}$ is naturally defined by

[TABLE]

As a result,

[TABLE]

We use Equation (4) to characterize the growth function of $\mathcal{F}$, defined by $\Pi_{\mathcal{F}}(m)=\max_{\mathcal{S}\in\mathcal{Z}^{m}}\left|\mathcal{F}\cap\mathcal{S}\right|$. We bound $\Pi_{\mathcal{F}}(m)$ using Lemma 3.1 and the Sauer-Shelah lemma.

Lemma 3.2**.**

$\Pi_{\mathcal{F}}(m)\leq(em)^{10\sum_{i=1}^{N}d_{i}}$.

Before we claim for uniform convergence in probability, we must relate the number of distinct profiles (i.e., elements in $\mathcal{W}\cap\mathcal{S}$) in the empirical game under $\mathcal{S}$ and the size of $\mathcal{F}\cap\mathcal{S}$. The following claim shows that the size of $\mathcal{F}\cap\mathcal{S}$ is an upper bound on the size of $\mathcal{W}\cap\mathcal{S}$.

Claim 3.3**.**

It holds that $\Pi_{\mathcal{W}}(m)\leq\Pi_{\mathcal{F}}(m)$.

Next, we bound the probability of a player $i$’s payoff being “too far” from its empirical counterpart. The proof of Lemma 3.4 below goes along the path of Vapnik and Chervonenkis [29]. Since in our case $\mathcal{F}$ is not a binary function class, a few modifications are needed.

Lemma 3.4**.**

Let $m$ be a positive integer, and let $\epsilon>0$. It holds that

[TABLE]

The following theorem bounds the probability that any player $i$ has a difference greater than $\epsilon$ between its payoff and its empirical payoff (over the selection of a sample $\mathcal{S}$), uniformly over all possible strategy profiles. This is done by simply applying the union bound on the bound already obtained in Lemma 3.4.

Theorem 3.5**.**

Let $m$ be a positive integer, and let $\epsilon>0$. It holds that

[TABLE]

3.2 Existence of a PNE in empirical games

In the previous subsection we bounded the probability of a payoff vector being too far from its counterpart in the empirical game. Notice, however, that this result implies nothing about the existence of a PNE or an approximate PNE: for a fixed $\mathcal{S}$, even if $\sup_{\boldsymbol{h}\in\mathcal{H}}\left|\pi_{i}(\boldsymbol{h})-\pi_{i}^{\mathcal{S}}(\boldsymbol{h})\right|<\epsilon$ holds for every $i$, a player may still have a beneficial deviation. Therefore, the results of the previous subsection are only meaningful if we show that there exists a PNE in the empirical game, which is the goal of this subsection. We prove this existence using the notion of potential games [20].

A non-cooperative game is called a potential game if there exists a function $\Phi:\mathcal{H}\rightarrow\mathbb{R}$ such that for every strategy profile $\boldsymbol{h}=(h_{1},\dots,h_{N})\in\mathcal{H}$ and every $i\in\mathcal{N}$, whenever player $i$ switches from $h_{i}$ to a strategy $h_{i}^{\prime}\in\mathcal{H}_{i}$, the change in her payoff function equals the change in the potential function, i.e.,

[TABLE]

Theorem 3.6** ([20, 23]).**

Every potential game with a finite strategy space possesses at least one PNE.

Obviously, in our setting the strategy space of a game instance $\langle\mathcal{Z},\mathcal{D},\mathcal{N},\mathcal{H},\pi\rangle$ is typically infinite. Infinite potential games may also possess a PNE (as discussed in [20]), but in our case the distribution $\mathcal{D}$ is approximated from samples and the empirical game is finite, so no stronger claims are needed. Lemma 3.7 below shows that every empirical game is a potential game.

Lemma 3.7**.**

Every empirical game $\langle\mathcal{Z},\mathcal{S}\sim\mathcal{D}^{m},\mathcal{N},\mathcal{H},\pi\rangle$ has a potential function.

As an immediate result of Theorem 3.6 and Lemma 3.7,

Corollary 3.8**.**

Every empirical game $\langle\mathcal{Z},\mathcal{S}\sim\mathcal{D}^{m},\mathcal{N},\mathcal{H},\pi\rangle$ possesses at least one PNE.

After establishing the existence of a PNE in the empirical game, we are interested in the rate with which it can be “learnt”. More formally, we are interested in the convergence rate of the dynamics between the players, where at every step one player deviates to one of her $\epsilon$-better responses. Such dynamics do not necessarily converge in general games, but do converge in potential games. By examining the specific potential function in our class of (empirical) games, we can also bound the number of steps until convergence.

Lemma 3.9**.**

Let $\langle\mathcal{Z},\mathcal{S}\sim\mathcal{D}^{m},\mathcal{N},\mathcal{H},\pi\rangle$ be any empirical game instance. After at most $O\left(\frac{\log N}{\epsilon}\right)$ iterations of any $\epsilon$-better-response dynamics, an $\epsilon$-PNE of the empirical game is obtained.

Remark: Due to Equation (1), we know that any beneficial deviation improves the payoff of the deviating player by at least $\frac{1}{mN}$; hence, Lemma 3.9 suggests that an (exact) empirical PNE could be obtained using at most $O\left(mN\log N\right)$ iterations of any $\frac{1}{mN}$-better-response dynamics.

3.3 Learning $\epsilon$-PNE with high probability

In this subsection we leverage the results of the previous Subsections 3.1 and 3.2 to devise Algorithm 1, which runs in polynomial time and returns an approximate equilibrium with high probability. More precisely, we show that Algorithm 1 returns an $\epsilon$-PNE with probability of at least $1-\delta$, and has time complexity of $\text{poly}\left(\frac{1}{\epsilon},m,N,\log\left(\frac{1}{\delta}\right),d\right)$. As in the previous subsections, we denote $d=\sum_{i=1}^{N}d_{i}$.

First, we bound the required sample size. Using standard algebraic manipulations on Equation (5), we obtain the following.

Lemma 3.10**.**

Let $\epsilon,\delta\in(0,1)$, and let

[TABLE]

With probability of at least $1-\delta$ over all possible samples $\mathcal{S}$ of size $m$, it holds that

[TABLE]

Given $\epsilon,\delta$, we denote by $m_{\epsilon,\delta}$ the minimal integer $m$ satisfying Equation (6). Lemma 3.10 shows that $m_{\epsilon,\delta}=O\left(\frac{d}{\epsilon^{2}}\log\left(\frac{d}{\epsilon^{2}}\right)+\frac{1}{\epsilon^{2}}\log\left(\frac{N}{\delta}\right)\right)$ are enough samples to have all empirical payoff vectors $\epsilon$-close to their theoretic counterpart coordinate-wise (i.e., in the $L^{\infty}$ norm), with a probability of at least $1-\delta$.

Next, we bind an approximate PNE in the empirical game with an approximate PNE in the (actual) game.

Lemma 3.11**.**

Let $m\geq m_{\frac{\epsilon}{4},\delta}$ and let $\boldsymbol{h}$ be an $\frac{\epsilon}{2}$-PNE in $\langle\mathcal{Z},\mathcal{S}\sim\mathcal{D}^{m},\mathcal{N},\mathcal{H},\pi\rangle$. Then $\boldsymbol{h}$ is an $\epsilon$-PNE with probability of at least $1-\delta$.

Recall that Lemma 3.9 ensures that every $O\left(\frac{\log N}{\epsilon}\right)$ iterations of any $\epsilon$-better-response dynamics must converge to an $\epsilon$-PNE of the empirical game. In each such iteration, a player calls her approximate better-response oracle, which is assumed to run in $\text{poly}(\frac{1}{\epsilon},m,N)$ time. Altogether, given $\epsilon$ and $\delta$, Algorithm 1 runs in $\text{poly}\left(\frac{1}{\epsilon},N,\log\left(\frac{1}{\delta}\right),d\right)$ time, and returns an $\epsilon$-PNE with probability of at least $1-\delta$.

Corollary 3.12**.**

Let $\langle\mathcal{Z},\mathcal{D},\mathcal{N},\mathcal{H},\pi\rangle$ be a game, $\epsilon,\delta\in(0,1)$, and let $\boldsymbol{h}$ be the output of Algorithm 1. With probability of at least $1-\delta$, $\boldsymbol{h}$ is an $\epsilon$-PNE.

4 Linear Strategy Space

Linear regression is extensively studied in the statistics/machine learning literature, and hence we find it the most appropriate use-case to examine in our competitive setting. In this section, we shall assume $\mathcal{H}_{i}$ is the linear strategy space for an arbitrary $i\in\mathcal{N}$. Formally, $\mathcal{H}_{i}$ is the function class of all linear mappings from $\mathcal{X}\subseteq\mathbb{R}^{n}$ to $\mathbb{R}$. For ease of notation, we shall treat every $h_{i}\in\mathcal{H}_{i}$ as a vector in $\mathbb{R}^{n}$ that corresponds to the mapping $x\mapsto\langle\,h_{i},x\rangle$ for $x\in\mathbb{R}^{n}$. 333We shall keep using non-bold notation for single strategies and input vectors, to distinguish between those and strategy profiles. Under this representation, the class $\mathcal{H}_{i}$ can be referred to as $\mathbb{R}^{n}$. We shall further assume that the dimension of the input $n$ is constant.

We first present an oracle for the best linear response. We describe informally how a player $i$ can compute a best response against any $\boldsymbol{h}_{-i}$, and then devise an algorithm that is inspired by the one proposed in Ben-Porat and Tennenholtz [6]. Algorithm 3 finds a best linear response against any strategies selected by the other players, not necessarily linear strategies, and would apply even if the other players employ more sophisticated algorithms, e.g., neural networks. Algorithm 3 runs in polynomial time in the size of the sample $m$. Afterwards, we apply Algorithm 1 along with the best response oracle implemented in Algorithm 3 on several synthetic datasets in a two-player game with linear strategies, and examine the equilibrium structure and properties.

4.1 A Polynomial Best-Response Oracle

Before we present the formal algorithm, we give a high-level intuition of how it works. Let $\mathcal{S}=(z_{j})_{j=1}^{m}$ be an arbitrary sample, $i\in\mathcal{N}$ be an arbitrary player index, and $\boldsymbol{h}_{-i}$ be arbitrary strategies of all players but $i$. Denote by $M$ the mapping $h_{i}\mapsto\boldsymbol{w}$ such that

[TABLE]

Observe that $M$ need not be onto, as not all vectors in $\{0,\frac{1}{N},\frac{2}{N},\dots 1\}^{m}$ may be in the image of $M$, nor one-to-one, as $M(h_{i})=M(h_{i}^{\prime})$ for $h_{i},h_{i}^{\prime}\in\mathcal{H}_{i},h_{i}\neq h_{i}^{\prime}$ may occur. In addition, the payoff of player $i$ under $h_{i}$ is precisely $\left\lVert M(h_{i})\right\rVert_{1}$. While the size of the target set of $M$ is exponential in $m$, Lemma 3.1 implies that the size of the image of $M$ is only polynomial in $m$. The idea is therefore to do an exhaustive search over the image of $M$ in polynomial time, find a vector $\boldsymbol{w}^{*}\in\operatorname*{arg\,max}_{\boldsymbol{w}\in\text{Image}(M)}\left\lVert\boldsymbol{w}\right\rVert_{1}$, and then find $h_{i}^{*}\in\mathcal{H}_{i}$ such that $M(h_{i}^{*})=\boldsymbol{w}^{*}$.

Next, we introduce an auxiliary decision problem that facilitates the search over the image of $M$, the Partial Vector Feasibility problem (PVF). For every strategy $h_{i}\in H_{i}$ and a point $(x,y,t)$, we focus on the term $\left|\langle\,h_{i},x\rangle-y\right|$. In case it is less or equal to $t$, player $i$ will get (some fraction) of this point. Otherwise, she will not, and two cases can occur: either $\langle\,h_{i},x\rangle-y$ is greater than $t$ (above $t$), or $\langle\,h_{i},x\rangle-y$ is less than $-t$ (below $t$). A PVF problem is a decision problem that asks whether a partial set of the points can be classified into a particular assignment of these three cases; it is formally defined in Algorithm 2.

Note that PVF is solvable in polynomial time via linear programming. We are now ready to present the Best Linear Response (BLR) algorithm. BLR has three main steps:

Compute the possible gain from each point in the sample. The possible gain from each point is a function of $\boldsymbol{h}_{-i}$. 2. 2.

Find all feasible subsets of points player $i$ can satisfy concurrently (i.e., vectors in $\mathcal{F}_{i}\cap\mathcal{S}$, where $\mathcal{F}_{i}$ is as defined in Equation (3)). 3. 3.

Return a strategy that achieves the highest possible payoff.

The first step consists of a straightforward computation. To motivate the second step, notice that if $\mathcal{I}(z_{j},h_{i})=0$, then either $\langle\,h_{i},x_{j}\rangle-y_{j}>t_{j}$ or $\langle\,h_{i},x_{j}\rangle-y_{j}<-t_{j}$ holds. Therefore, we identify all vectors $\boldsymbol{v}=(v_{1},\dots v_{m})\in\{1,a,b\}^{m}$ such that there exists $h_{i}\in\mathcal{H}_{i}$ and $v_{j}=1$ if $\mathcal{I}(z_{j},h_{i})=1$; $v_{j}=a$ if $\langle\,h_{i},x_{j}\rangle-y_{j}>t_{j}$ (“above”); and $v_{j}=b$ if $\langle\,h_{i},x_{j}\rangle-y_{j}<-t_{j}$ (“below”). This is done by recursively partitioning $\{1,a,b\}^{m}$, where in each iteration we consider only a prefix of the entries, while the suffix is masked with “0” (see PVF). At the end of this step, we have fully identified the set $\mathcal{F}_{i}\cap\mathcal{S}$, since every vector in $\{1,a,b\}^{m}$ can be mapped to a vector in $\mathcal{F}_{i}\cap\mathcal{S}$ by replacing $a,b$ with [math]. Due to the bijection between $\mathcal{F}_{i}\cap\mathcal{S}$ and the image of $M$ (see Subsection 3.1), we have essentially discovered the image of $M$; thus, we can pick a feasible vector that corresponds to the highest payoff. Finally, we find a strategy that attains the highest payoff by invoking PVF for the last time. The above discussion is formulated via Algorithm 3.

Recall that we assume that the input space $n$ is constant. Ben-Porat and Tennenholtz [6] show that the second step (the for loop in line 3) is done in time $\text{poly}(m)$, and that $\mathcal{R}_{m}$ is of polynomial size. The first step (line 3) and the last step (lines 3 and 3) are clearly executed in polynomial time. Overall, Algorithm 3 runs in polynomial time. In addition, since it considers all possible distinct strategies using $\mathcal{F}_{i}\cap\mathcal{S}$ and takes the one with the highest payoff, it indeed returns the best linear response with respect to $\boldsymbol{h}_{-i}$ in the empirical game. To sum,

Theorem 4.1**.**

Let $\mathcal{H}_{i}$ be the linear strategy space, $\mathcal{H}_{-i}$ be the product of any strategy spaces, $\boldsymbol{h}_{-i}$ be an element from $\mathcal{H}_{-i}$ and let $\mathcal{S}$ be a sample of size $m$. Algorithm 3 finds $h_{i}^{*}$ such that $h_{i}^{*}\in\operatorname*{arg\,max}_{h_{i}\in\mathcal{H}_{i}}\pi_{i}^{\mathcal{S}}(\boldsymbol{h})$ in time $\text{poly}(m)$.

We note that the best response need not be unique, as Algorithm 3 employs two tie breakers. The first tie breaker is the selection of a feasible vector $\boldsymbol{v}^{*}$ with the highest payoff in Line 3, since several such vectors may exist. Second, the call $\textsc{PVF}(\mathcal{S},\boldsymbol{v}^{*})$ in Line 3 returns a single strategy, albeit the linear program it solves probably possesses infinitely many optimal solutions.

4.2 Simulations with Synthetic Data in $\mathbb{R}^{2}$

The goal of this subsection is to visualize equilibrium profiles in several scenarios, and analyze the implications of different tolerance levels on the equilibrium structure. In service of that, we focus on two-player games with $\mathcal{X}\times\mathcal{Y}=\mathbb{R}^{2}$, and the linear strategy space for both players. Recall that an equilibrium profile can be obtained by executing Algorithm 1, where Algorithm 3 plays the role of a best response (which is a special case of $\epsilon$-better response). Since our main purpose is to discuss the structure of equilibria, we focus solely on the empirical games and completely neglect the generalization analysis. Note, however, that the results obtained in the previous sections suggest that the payoffs in equilibrium will be similar to their empirical counterparts.

We now describe the high-level details of the synthetic datasets we employ. We let the instance $x$ be uniformly distributed in a close segment. In addition, we consider four conditions for the relations between the instance $x$ and the value $y$:

Linear: for every instance $x$, its value $y$ is a linear function of $x$, plus an additive normal noise. Namely, $y=ax+b+\epsilon$ for $a,b\in\mathbb{R}$ and $\epsilon\sim Normal(0,1)$. 444$a$ and $b$ are not related to the arbitrary numbers used in Algorithm 2. 2. 2.

V-shape: the value of every $x$ is determined by $y=a\left|x-x_{0}\right|+b+\epsilon$ for $x_{0},a,b\in\mathbb{R}$ and $\epsilon\sim Normal(0,1)$. 3. 3.

X-shape: for every instance $x$, the value $y$ is distributed as follows:

[TABLE]

where $a_{1},b_{1},a_{2},b_{2}\in\mathbb{R}$ and $\epsilon\sim Normal(0,1)$. 4. 4.

Piecewise: for every $x$, the value $y$ is given by

[TABLE]

where $x_{0},a_{1},b_{1},a_{2},b_{2}\in\mathbb{R}$ and $\epsilon\sim Normal(0,1)$.

We have so far described the distribution over $\mathcal{X}\times\mathcal{Y}$ for all four conditions. A visualization of generated datasets according to these conditions appears in Figure 2. To complete a game description, we need to specify the tolerance levels. Different tolerance levels alter the equilibrium behavior, and thus are interesting to explore. In these simulations, we use a constant tolerance level, i.e., given a set $(x_{j},y_{j})_{j=1}^{m}$ associated with one of the four conditions above and a value $t\in\mathbb{R}$, the dataset we analyze is $(x_{j},y_{j},t)_{j=1}^{m}$. By doing so, we do not exploit the full generality of the model and algorithms. Nevertheless, since the tolerance levels are non-obvious to simulate, we feel that this is justified. To examine the extent to which the payoffs are sensitive to the tolerance levels, we consider three levels of tolerance, which vary across the four conditions above.

We generated555The specifics are elaborated in Section D. Our code is available at: https://github.com/omerbp/Regression-Equilibrium. 12 datasets, composed of the four conditions and three tolerance levels for each condition. For each dataset, we run Algorithm 1, with Algorithm 3 playing the role of a best response oracle, until convergence and observed the obtained equilibrium profile.

Our findings are reported in Figure 3. Each column of sub-figures corresponds to one of the four conditions above. The rows correspond to tolerance levels in a decreasing order. For instance, the top leftmost sub-figure represents the linear condition with high tolerance level. The colored lines are the strategies of the players in a PNE profile, as obtained by Algorithm 1. A point is circled-red (square-blue) if only the dashed-red (solid-blue) player produces an accurate prediction for it. Alternatively, a point is star-green if both players produce an accurate prediction for it, and x-black if both predictions are inaccurate (i.e., point $j$ is black $\left|h_{i}(x_{j})-y_{j}\right|>t$ for $i\in\{1,2\}$ and the corresponding tolerance level $t$).

Observe that the players’ strategies are distinguishable, even for the linear condition. Noticeably, the number of star-green points decreases with the tolerance level. It symbolizes that the number of points obtaining an accurate prediction from both players decreases with the tolerance. In contrast, the number of x-black points increases with the tolerance, as less points get an accurate prediction when the tolerance is low. Moreover, notice that a high tolerance allows the players to gain almost all points, but when the tolerance is low roughly half of the points obtain an inaccurate prediction. As for player payoffs, the two players get almost the same payoff in all datasets, albeit theoretically the payoffs may greatly differ; this is due to the symmetric nature of the datasets.

5 Extensions

In this section we answer two interesting questions. First, in Subsection 5.1, we deal with an infinite pseudo-dimension. While we show learning may not occur, similarly to other machine learning applications (see, e.g., [25]), we obtain this result even if the players fully observe $\mathcal{D}$. This conclusion is in sheer contrast to classical results, since when $\mathcal{D}$ is observed there is essentially nothing to learn. In our setting, however, equilibrium strategy is another object that has to be learned. Second, we analyze a variant of the model that is aligned with the payoff function of Ben-Porat and Tennenholtz [6] and Immorlica et al. [16], Namely, where each user (associated with an example) grants one monetary unit to the player who provides the most accurate prediction. Unlike the existence we proved in Subsection 3.2 for the main model, under this variant an empirical PNE may not exist.

5.1 Learnability in games with infinite dimension

While Lemma 3.1 upper bounds the VC dimension of $\mathcal{F}_{i}$, the following Claim 5.1 puts a lower bound on it. Claim 5.1 implies that if $\textnormal{Pdim}(\mathcal{H}_{i})$ is infinite, so is $\textnormal{VCdim}(\mathcal{F}_{i})$.

Claim 5.1**.**

$\textnormal{VCdim}(\mathcal{F}_{i})\geq\textnormal{Pdim}(\mathcal{H}_{i})$.

Classical results in learning theory suggest that if $\textnormal{VCdim}(\mathcal{F}_{i})=\infty$, a best response on the sample may not generalize to an approximate best response w.h.p. To see this, imagine a “game” with one player, who seeks to maximize her payoff function. No Free Lunch Theorems (see, e.g., [30]) imply that with a constant probability, the player cannot get her payoff within a constant distance from the optimal payoff. We conclude that in general games, if a player has a strategy space with an infinite pseudo-dimension, she may not be able to learn. However, in the presence of such a player, can other players with a finite pseudo-dimension learn an approximate best-response? In our setting, players are interacting with each other, and player payoffs are a function of the whole strategy profile; thus, different challenges may arise.

The strategy space of a player $i$ in an empirical game induced by a sample $\mathcal{S}$ is $\mathcal{H}_{i}\cap\mathcal{S}$, namely the restriction of her strategy space to the sample. Due to the structure of the payoff function, player $i$ can compute an empirical best response to $\boldsymbol{h}_{-i}\cap\mathcal{S}$ without knowing $\boldsymbol{h}_{-i}$ nor $\boldsymbol{\mathcal{H}}_{-i}$. As the sample size increases $\mathcal{S}$ becomes a representative of $\mathcal{D}$, and we employed uniform convergence to conclude that w.h.p. $\mathcal{H}\cap\mathcal{S}$ is “similar” to $\mathcal{H}$; hence, players use the sample to learn both the underline distribution and a best response to the other players. In the rest of this subsection we give an example where knowledge about the strategy spaces of the other players is crucial. In particular, we show that if player $i$ (for $i=1$) has a strategy space with infinite dimension, she can trick the other players and drive them to her preferable outcome.

Example 5.1**.**

Let $\mathcal{D}$ be a density function over $\mathcal{Z}=[0,2]\times[0,1]\times\left\{\frac{1}{2}\right\}$ as follows:

[TABLE]

We now define several strategies that will constitute the strategy spaces. Let $h^{0}(x)\equiv 0,h^{1}(x)\equiv 1$ be constant functions mapping each instance $x$ to 0, 1 correspondingly. Moreover, for any $\mathcal{S}\subset\mathcal{Z}$, denote

[TABLE]

In other words, $h^{\mathcal{S}\rightarrow 0}$ labels 0 (and $h^{\mathcal{S}\rightarrow 1}$ labels 1) every instance $x$ that appears in the projection of $\mathcal{S}$ on its first entry; otherwise, it labels it correctly according to $\mathcal{D}$, i.e., $h^{\mathcal{S}\rightarrow 0}(x)=\mathds{1}_{1\leq x\leq 2}$ (and similarly, $h^{\mathcal{S}\rightarrow 1}(x)=\mathds{1}_{1\leq x\leq 2}$). Let $\mathcal{N}=\{1,2,3\}$, $\mathcal{H}_{2}=\mathcal{H}_{3}=\{h^{0},h^{1}\}$, and denote

[TABLE]

Notice that $\mathcal{H}_{1}$ is “complex”, i.e., $\textnormal{Pdim}(\mathcal{H}_{1})=\infty$. Next, we define a game $\mathcal{G}$ such that $\mathcal{G}=\langle\mathcal{Z},\mathcal{D},\mathcal{N},\mathcal{H},\pi\rangle$ for $\mathcal{H}=\mathcal{H}_{1}\times\mathcal{H}_{2}\times\mathcal{H}_{3}$. Denote by $\boldsymbol{h}$ the strategy profile $\boldsymbol{h}=(h^{\mathcal{S}\rightarrow 0},h^{1},h^{1})$, and by $\boldsymbol{h}^{\prime}$ the strategy profile $\boldsymbol{h}^{\prime}=(h^{0},h^{1},h^{1})$. Observe that $\pi_{1}(\boldsymbol{h})=\frac{2}{3},\pi_{2}(\boldsymbol{h})=\pi_{3}(\boldsymbol{h})=\frac{1}{6}$, while $\pi_{1}(\boldsymbol{h}^{\prime})=\frac{1}{2},\pi_{2}(\boldsymbol{h}^{\prime})=\pi_{3}(\boldsymbol{h}^{\prime})=\frac{1}{4}$. Moreover, $h^{\mathcal{S}\rightarrow 0}$ is a best response of player 1 to $\boldsymbol{h}_{-1}$, while $h^{1}$ is a best response of player 2 to $\boldsymbol{h}^{\prime}_{-2}$ (similarly for player 3). As a result, player 1 would like to play $h^{\mathcal{S}\rightarrow 0}$ against $\boldsymbol{h}_{-1}$, but let player 2 and 3 think she plays $h^{0}$ and therefore mislead them to assume that playing $h^{1}$ (as they do under $\boldsymbol{h}^{\prime}$) is a best response.

Since $h^{0}$ and $h^{\mathcal{S}\rightarrow 0}$ coincide on every sample $\mathcal{S}$, players 2 and 3 cannot distinguish between $\boldsymbol{h}$ and $\boldsymbol{h}^{\prime}$ by observing the empirical payoffs. Moreover, with probability of at least $\frac{1}{4}$ over all choices of $\mathcal{S}$ for $\left|\mathcal{S}\right|\geq 15$ we have $\frac{1}{2}<\frac{1}{m}\sum_{j=1}^{m}\mathds{1}_{y_{j}=1}<\frac{3}{4}$ (see Claim C.2 in the appendix). Under such a sample $\mathcal{S}$, player 2 and 3 indeed play a best response under $\boldsymbol{h}^{\prime}$ in the empirical game induced by $\mathcal{S}$ (while player 1 is not). Consequently, their lack of knowledge about $\mathcal{H}_{1}$ forbids them to better-respond to player 1 and leads them to sub-optimal payoffs (each gets $\frac{1}{6}$ compared to $\frac{1}{4}$ if she deviates) with constant probability. By playing $h^{\mathcal{S}\rightarrow 0}$, which is sub-optimal in the empirical game of $\mathcal{G}$ on such $\mathcal{S}$ (since $\pi_{1}^{\mathcal{S}}(h^{\emptyset\rightarrow 0},\boldsymbol{h}_{-1})-\pi_{1}^{\mathcal{S}}(\boldsymbol{h})>\frac{1}{3}$), player 1 can manipulate the others to think that she plays $h^{0}$.

The intuition behind the example is as follows. Despite $\mathcal{D}$ being relatively simple, the strategy selected by player $1$ is “complex”, in the sense that its behavior on the sample and on the population is substantially different; thus, players $2$ and $3$ are only playing a best response to the observable, simple way player $1$ is playing on the sample, yet misinterpret the extension of player $1$’s strategy to the population. The results of the previous sections imply that this phenomena cannot occur when the pseudo dimensions are finite, as long as the sample is large enough. Another interesting point is that in Example 5.1 each player can find a strategy that maximizes her payoff if she were alone using a small number of samples. Indeed, this inability to generalize from samples follows solely from strategic behavior. Notice that if player 2 has knowledge of $\mathcal{H}_{1}$, she can infer that her strategy under $\boldsymbol{h}^{\prime}$ is sub-optimal. However, knowledge of the strategy spaces of other players is a heavy assumption: the better-response dynamics we discussed in Subsection 3.2 only assumed that each player can compute a better response.

5.2 Non-existence of PNE in the Direct Attraction Model

In this subsection we consider a variant of the model, where a player payoff is the proportion of users for whom she provides the most accurate prediction. Notice that this payoff scheme is aligned with the one considered by Immorlica et al. [16]. We show a negative result for this payoff scheme – a PNE may not exist without further assumptions on $\mathcal{D}$. Surprisingly, this is true even when players are restricted to linear strategies. Finally, we highlight a special case where a PNE does exist. To facilitate understanding, we denote the elements we reconsider in this subsection by a tilde.

First, we formalized the payoff scheme considered in this subsection. Let $\tilde{\mathcal{Z}}=\mathcal{X}\times\mathcal{Y}$, $\tilde{\mathcal{D}}$ be the marginal distribution of $\mathcal{D}$ over $\tilde{\mathcal{Z}}$ and let $\tilde{\mathcal{S}}$ denote a sample from $\tilde{\mathcal{Z}}$. Given $\tilde{z}=(x,y)\in\tilde{\mathcal{Z}}$ and a strategy profile $\boldsymbol{h}$, Denote by $B(\tilde{z};\boldsymbol{h})$ the number of players who provide the most accurate prediction. In other words,

[TABLE]

In addition, let

[TABLE]

Finally, the payoffs and the empirical payoffs are defined by

[TABLE]

Indeed, this payoff depicts the case where each user is attracted to the player (or some, if there is more than one) that offered her the most accurate prediction. The induced game is a constant sum game, as now every user (or a point in $\tilde{\mathcal{Z}}$) grants a monetary unit to one of the players w.p. 1. Moreover, notice that the third entry of $\mathcal{Z}$, the tolerance, is completely disregarded in this payoff scheme.

Recall that in Section 3 we have demonstrated that players can use a sample drawn i.i.d. to learn an approximate PNE with high probability. However, as we show next, under the revised payoff scheme a PNE may not exist.

Proposition 5.2**.**

A game $\langle\tilde{\mathcal{Z}},\tilde{\mathcal{D}},\mathcal{N},\mathcal{H},\tilde{\pi}\rangle$ may not possess an approximate PNE for a constant approximation factor.

Proposition 5.2 implies that under the payoff scheme defined in Equation (8), players may not learn an $\epsilon$-PNE from a sample, regardless of the sample size. We prove Proposition 5.2 by providing a special case, which is given in Example 5.2.

Example 5.2**.**

Let $\tilde{\mathcal{Z}}\subset\mathbb{R}^{2}$, and let $U\subset\tilde{\mathcal{Z}}$ be a union of squares as follows:

[TABLE]

See Figure 4 for illustration. Let $\tilde{\mathcal{D}}$ be the uniform distribution over $U$, let $N=2$ (i.e., a two-player game) and let $\mathcal{H}_{1}=\mathcal{H}_{2}$ be the linear strategy space. Observe that

Claim 5.3**.**

There exists no $\left(\frac{1}{6}-\epsilon\right)$-PNE in $\langle\tilde{\mathcal{Z}},\tilde{\mathcal{D}},\mathcal{N},\mathcal{H},\tilde{\pi}\rangle$, for any arbitrarily small constant $\epsilon>0$.

The above Claim 5.3 proves Proposition 5.2.

In fact, it is not immediate that the game proposed in Example 5.2 even possesses a mixed Nash equilibrium, despite being a symmetric zero-sum two-player game. Observe that not only the strategy space of the players is infinite, but also player payoffs are discontinuous, so Glicksberg’s theorem [12] cannot be applied. Since this work focuses on pure strategy profiles, we leave the full analysis of this extension for future work.

Before the end of this section, we highlight a class of games that do possess a PNE. The setting depicted in Example 5.2 is carefully crafted to demonstrate the negative effects of $\tilde{\pi}$ as defined in Equation (8). Arguably, linear strategies are bad predictors for the structure of the data in this example, which is highly non-linear. One may ask whether a PNE must exist in the realizable case.

Definition 5.1**.**

A game $\langle\tilde{\mathcal{Z}},\tilde{\mathcal{D}},\mathcal{N},\mathcal{H},\tilde{\pi}\rangle$ is realizable if for every $i\in\mathcal{N}$ and $(x,y)\in\tilde{\mathcal{Z}}$ there exists $h_{i}^{*}$ such that $h_{i}^{*}(x)=y$.

In a realizable game, every player has a strategy that predicts the label of every instance correctly. Clearly, $(h_{1}^{*},\dots,h_{N}^{*})$ is a PNE of the game, and so at least one PNE exists. We now show that an approximate PNE of a realizable game can be found using the technique we developed in Section 3. Given $\epsilon,\delta$, let $m=m_{\frac{\epsilon}{2},\delta}$ that satisfies Equation (6), and let $\tilde{\mathcal{S}}$ be a sample of size $m$ from $\tilde{\mathcal{D}}$. Due to the realizability assumption, every player $i$ has a strategy that accurately predicts the value of every instance in the sample; let $h_{i}$ be one such strategy (this does not imply that $h_{i}=h_{i}^{*}$). Importantly, $h_{i}$ can be found using the same best response oracle employed earlier in Algorithm 1, when augmenting every point in $\tilde{\mathcal{S}}$ to have 0 in its third entry (i.e., zero tolerance). The above discussion is formalized via Algorithm 4. We show that it outputs an $\epsilon$-PNE with probability of at least $1-\delta$.

Lemma 5.4**.**

Let $\langle\tilde{\mathcal{Z}},\tilde{\mathcal{D}}^{m},\mathcal{N},\mathcal{H},\tilde{\pi}\rangle$ be a realizable game, $\epsilon,\delta\in(0,1)$, and let $\boldsymbol{h}$ be the output of Algorithm 4. With probability of at least $1-\delta$, $\boldsymbol{h}$ is an $\epsilon$-PNE.

6 Discussion

We have presented a novel setting that deepens the intersection between learning theory and game theory. We used the seminal framework of Valiant to model an environment where users seek predictions and players provide them, where players’ payoffs are defined in the spirit of Dueling Algorithms [16]. We proved the existence of a PNE, bounded the number of steps until convergence and formalized an algorithm that computes a PNE assuming better-response oracles. We provided an oracle for linear strategy space, and demonstrated the equilibrium structure for linear two-player games in several settings. We have also considered cases where players cannot learn a PNE, despite that user distribution is fully known, and provided preliminary results on an interesting variant of the model, which will appear in future work.

As mentioned in Section 2.1, our analysis assumes players have better-response oracles. In fact, our model and results are valid for a much more general scenario, as described next. Consider the case where players only have heuristics for finding a better response. After running heuristic better-response dynamics and obtaining a strategy profile, the uniform convergence property implies that the payoffs with respect to the whole population are guaranteed to be close to their empirical counterparts, w.h.p.; therefore, our analysis is still meaningful even if players cannot maximize their empirical payoff efficiently. The bounds on the required sample size we obtained in Section 3 and the rate of convergence are relevant for this case as well.

Ultimately, we remark that regression equilibrium can be employed to develop piece-wise linear models, or other mixture models in general, as illustrated in Subsection 4.2 and Figure 3. This is a future direction of independent interest.

Acknowledgments

We thank the anonymous reviewers for their helpful comments. Example 5.2 is a simplified version of the one we had originally, and is due to Yakov Babichenko. The work of O. Ben-Porat was supported by a JPMorgan Chase & Co. PhD Fellowship. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement n$\degree$ 740435).

Appendix A Proof of Lemma 3.1

Proof of Lemma 3.1.

First, we define two auxiliary classes of binary functions $\mathcal{G}^{\geq},\mathcal{G}^{\leq}$ such that

[TABLE]

Claim A.1**.**

$\textnormal{VCdim}(\mathcal{G}^{\geq})=\textnormal{VCdim}(\mathcal{G}^{\leq})=d_{i}$.

The proof of Claim A.1 appears after the proof of this lemma. Next, we wish to bound the growth function of $\mathcal{F}_{i}$ using the growth function of $\mathcal{G}^{\geq}$ and $\mathcal{G}^{\leq}$.

Claim A.2**.**

$\Pi_{\mathcal{F}_{i}}(m)\leq\Pi_{\mathcal{G}^{\geq}}(m)\cdot\Pi_{\mathcal{G}^{\leq}}(m)$.

The proof of Claim A.2 appears after the proof of this lemma. We are now ready for the final argument. By the Sauer-Shelah lemma we know that every $m$ satisfying $2^{m}>\Pi_{\mathcal{F}_{i}}(m)$ is an upper bound on $\textnormal{VCdim}(\mathcal{F}_{i})$ [24]. In particular, for $m=10d_{i}$ we have

[TABLE]

therefore,

[TABLE]

∎

Proof of Claim A.1.

We prove the claim for $\mathcal{G}^{\geq}$, and by symmetric arguments one can show it holds for $\mathcal{G}^{\leq}$ as well.

Since $\textnormal{Pdim}(\mathcal{H}_{i})=d_{i}$, for every $m\leq d_{i}$ there is a sample $\mathcal{S}=(x_{1},\dots x_{m})\in\mathcal{X}^{m}$ and a witness $\boldsymbol{r}=(r_{1},\dots r_{m})\in\mathbb{R}^{m}$ such that for every binary vector $\boldsymbol{b}\in\{-1,1\}^{m}$ there is a function $h_{\boldsymbol{b}}$ for which $\operatorname{sign}(h_{\boldsymbol{b}}(x_{j})-r_{j})=b_{j}$ for all $j\in[m]$. Denote

[TABLE]

and focus on a particular $\boldsymbol{b}\in\{-1,1\}^{m}$. For every $j\in[m]$ such that $b_{j}=1$ we have

[TABLE]

In addition, if $b_{j}=-1$ then

[TABLE]

This is true for every $\boldsymbol{b}$; therefore, we showed that $\mathcal{G}^{\geq}$ shatters $\mathcal{S}^{\prime}$.

In the opposite direction, assume by contradiction that $\mathcal{G}^{\geq}$ shatters $\mathcal{S}^{\prime}=\left((x_{1},r_{1}),\dots,(x_{m},r_{m})\right)$ for $m\geq d_{i}+1$. Let $H=\{h_{\boldsymbol{b}}\}_{\boldsymbol{b}\in\{-1,1\}}\subset\mathcal{H}_{i}$ be a set of functions such that for every $\boldsymbol{b}$ there exists exactly one function $h_{\boldsymbol{b}}\in H$ satisfying $\mathds{1}_{h_{\boldsymbol{b}}(x)\geq r}(x_{j},r_{j})=1$ if $b_{j}=1$ and $\mathds{1}_{h_{\boldsymbol{b}}(x)\geq r}(x_{j},r_{j})=0$ if $b_{j}=0$. Notice that by definition of the VC dimension, such an $H$ must exist, and that $\left|H\right|=2^{m}$.

One cannot claim directly that $\mathcal{H}_{i}$ pseudo-shatters $\mathcal{S}=(x_{1},\dots,x_{m})$ with witness $\boldsymbol{r}=(r_{1},\dots r_{m})$, since $h_{\boldsymbol{b}}(x_{j})=r_{j}$ may hold for $b_{j}=1$, but we need $h_{\boldsymbol{b}}(x_{j})$ to be strictly greater than $r_{j}$; therefore, we construct a new witness: let $a_{j}$ be such that

[TABLE]

Notice that $\left|H\right|$ is finite so the maximum is well defined. In addition, $a_{j}<r_{j}$ since $h_{\boldsymbol{b}}(x_{j})<r_{j}$ for every $\boldsymbol{b}$ such that $b_{j}=-1$ (recall that if $b_{j}=0$ then $\mathds{1}_{h_{\boldsymbol{b}}(x)\geq r}(x_{j},r_{j})=0$ ).

Denote $\boldsymbol{r}^{*}=\frac{\boldsymbol{a}+\boldsymbol{r}}{2}$. Next, we claim that $\mathcal{H}_{i}$ pseudo-shatters $\mathcal{S}$ with the witness $\boldsymbol{r}^{*}$. Fix $\boldsymbol{b}\in\{-1,1\}^{m}$. If $b_{j}=-1$,

[TABLE]

On the other hand, if $b_{j}=1$, we have

[TABLE]

Combining these two equations, we get that $\operatorname{sign}(h_{\boldsymbol{b}}(x_{j})-r_{j}^{*})=b_{j}$ for all $j\in[m]$. Consequently, $\mathcal{H}_{i}$ pseudo-shatters $\mathcal{S}$ with witness $\boldsymbol{r}^{*}$; hence we obtained a contradiction.

Overall, we showed that $\textnormal{VCdim}(\mathcal{G}^{\geq})\geq d_{i}$ and $\textnormal{VCdim}(\mathcal{G}^{\geq})\leq d_{i}$; hence $\textnormal{VCdim}(\mathcal{G}^{\geq})=d_{i}$. ∎

Proof of Claim A.2.

Denote by $\mathcal{S}=(x_{j},y_{j},t_{j})_{j=1}^{m}\in\mathcal{Z}^{m}$ an arbitrary sample, and let $\mathcal{F}_{i}\cap\mathcal{S}$ be the restriction of $\mathcal{F}_{i}$ to $\mathcal{S}$. Formally,

[TABLE]

In addition, denote by $G^{\geq}$ the restriction of $\mathcal{G}^{\geq}$ to $(x_{j},y_{j}-t_{j})_{j=1}^{m}$, and similarly let $G^{\leq}$ be the restriction of $\mathcal{G}^{\leq}$ to $(x_{j},y_{j}+t_{j})_{j=1}^{m}$. We now show a one-to-one mapping $M:{\mathcal{F}_{i}}\cap{\mathcal{S}}\rightarrow G^{\geq}\times G^{\leq}$, implying that

[TABLE]

holds, thereby proving the assertion. Notice that for every $f_{h}\in\mathcal{F}_{i}$ such that $f_{h}(z)=1$ we have $\mathcal{I}(z,h)=1$ for the corresponding $h\in\mathcal{H}_{i}$; thus

[TABLE]

Alternatively, if $f_{h}(z_{j})=0$, we have $\mathcal{I}(z_{j},h)=0$ and

[TABLE]

By Equations (13) and (A) we have

[TABLE]

We define the mapping $M$ such that every vector $\left(\mathcal{I}(z_{1},h_{1}),\dots,\mathcal{I}(z_{m},h_{1})\right)\in\mathcal{F}_{i}\cap\mathcal{S}$ is mapped to

[TABLE]

Namely, every vector obtained by applying $f_{h}$ on the sample $\mathcal{S}$ is mapped to the vector formed by concatenating the two corresponding (same $h$) vectors from $G^{\leq}$ and $G^{\geq}$. Let $\boldsymbol{b}^{1},\boldsymbol{b}^{2}\in{\mathcal{F}_{i}}\cap\mathcal{S}$ such that $b^{1}_{j}\neq b^{2}_{j}$ for at least one index $j\in[m]$, and w.l.o.g. let $b^{1}_{j}=1$. Since $b^{1}_{j}=f_{h}(z_{j})=\mathcal{I}(z_{j},h)$, Equation (15) implies that $M(\boldsymbol{b}^{1})_{j}=M(\boldsymbol{b}^{1})_{j+m}=1$, while at least one of $\{M(\boldsymbol{b}^{2})_{j},M(\boldsymbol{b}^{2})_{j+m}\}$ equals zero; thus $M(\boldsymbol{b}^{1})\neq M(\boldsymbol{b}^{2})$. Hence $M$ is an injection.

Ultimately, notice that $\mathcal{S}$ is arbitrary; thus

[TABLE]

∎

Claim A.3**.**

$\left(\frac{n}{k}\right)^{k}\leq{n\choose k}\leq\left(\frac{en}{k}\right)^{k}$.

Proof of Claim A.3.

We prove the two claims separately.

$\bullet$ $\left(\frac{n}{k}\right)^{k}\leq{n\choose k}$: fix $n$. We prove by induction for $k\leq n$. The assertion holds for $k=1$. For $k\geq 2$ and every $m$ such that $0<m<k\leq$ we have

[TABLE]

thus

[TABLE]

$\bullet$ ${n\choose k}\leq\left(\frac{en}{k}\right)^{k}$: since $e^{k}=\sum_{i=0}^{\infty}\frac{k^{i}}{i!}$ (the Taylor expansion of $e^{k}$), we have $e^{k}>\frac{k^{k}}{k!}$; thus, $\frac{1}{k!}<\left(\frac{e}{k}\right)^{k}$. As a result,

[TABLE]

∎

Appendix B Omitted proofs

Proof of Lemma 3.2.

Recall that the Sauer-Shelah lemma implies that $\Pi_{\mathcal{F}_{i}}(m)\leq(em/d_{i})^{d_{i}}$ for $m>d_{i}+1$. Since $\left|\mathcal{F}\cap\mathcal{S}\right|=\prod_{i=1}^{N}\left|\mathcal{F}_{i}\cap\mathcal{S}\right|$, we have

[TABLE]

∎

Proof of Claim 3.3.

To prove the claim, we show a one-to-one function from $\mathcal{W}\cap\mathcal{S}$ to $\mathcal{F}\cap\mathcal{S}$ for any arbitrary $\mathcal{S}$. In particular, it is sufficient to show a mapping $w(z;\boldsymbol{h})\mapsto\mathcal{I}(z,\boldsymbol{h})$. Let $M$ be a mapping from $\{1,\frac{1}{2},\dots,\frac{1}{N},0\}^{N}$ to $\{0,1\}^{N}$ defined by

[TABLE]

For instance, if $N=4$ and $\boldsymbol{v}=(\frac{1}{2},\frac{1}{2},0,0)$, then $M(\boldsymbol{v})=(1,1,0,0)$. Clearly, $M$ is a one-to-one mapping from $\text{Image}(w(z;\boldsymbol{h}))$ to $\text{Image}(\mathcal{I}(z,\boldsymbol{h}))$. For every vector $\boldsymbol{w}\in\mathcal{W}\cap\mathcal{S}$, there exists $\boldsymbol{h}_{\boldsymbol{w}}\in\mathcal{H}$ such that

[TABLE]

By applying $M$ on every entry of $\boldsymbol{w}$, we obtain $\boldsymbol{v}$ such that

[TABLE]

Ultimately, since $\boldsymbol{h}_{\boldsymbol{w}}\in\mathcal{H}$, we conclude that $\boldsymbol{v}\in\mathcal{F}\cap\mathcal{S}$.

In fact, by considering $M^{-1}$, $M^{-1}:\{0,1\}^{N}\rightarrow\{1,\frac{1}{2},\dots,\frac{1}{N},0\}^{N}$ such that for every $\boldsymbol{u}\in\{0,1\}^{N}$,

[TABLE]

one can show that $\Pi_{\mathcal{F}}(m)\leq\Pi_{\mathcal{W}}(m)$ holds as well; therefore, $\Pi_{\mathcal{W}}(m)=\Pi_{\mathcal{F}}(m)$. ∎

Proof of Lemma 3.4.

The proof follows closely the four steps in the proof of the classical uniform convergence theorem for binary functions (see, e.g., [3, 28]). The only steps that need modification are Steps 3 and 4, but we present the full proof for completeness.

Step 1 – Symmetrization: First, we want to show that

[TABLE]

For each $\mathcal{S}$, let $\tilde{\boldsymbol{h}}(\mathcal{S})$ be a function for which $\left|\pi_{i}(\tilde{\boldsymbol{h}}(\mathcal{S}))-\pi_{i}^{\mathcal{S}}(\tilde{\boldsymbol{h}}(\mathcal{S}))\right|\geq\epsilon$ if such a function exists, and any other fixed function in $\mathcal{H}$ otherwise. Notice that if $\left|\pi_{i}(\tilde{\boldsymbol{h}}(\mathcal{S}))-\pi_{i}^{\mathcal{S}}(\tilde{\boldsymbol{h}}(\mathcal{S}))\right|\geq\epsilon$ and $\left|\pi_{i}(\tilde{\boldsymbol{h}}(\mathcal{S}))-\pi_{i}^{\mathcal{S}^{\prime}}(\tilde{\boldsymbol{h}}(\mathcal{S}))\right|\leq\frac{\epsilon}{2}$, then $\left|\pi_{i}^{\mathcal{S}}(\tilde{\boldsymbol{h}}(\mathcal{S}))-\pi_{i}^{\mathcal{S}^{\prime}}(\tilde{\boldsymbol{h}}(\mathcal{S}))\right|\geq\frac{\epsilon}{2}$ (triangle inequality); thus,

[TABLE]

since $\mathcal{S},\mathcal{S}^{\prime}$ are independent and due to Claim C.1.

Step 2 – Permutations: We denote $\Gamma_{2m}$ as the set of all permutations of $[2m]$ that swap $i$ and $m+i$ in some subset of $[m]$. Namely,

[TABLE]

where $\Pi([2m])$ denotes the set of permutations over $[2m]$. In addition, for $\mathcal{S}=(z_{1},\dots,z_{2m})$, let $\sigma(\mathcal{S})=(z_{\sigma(1)},\dots,z_{\sigma(2m)})$. Notice that for every $\sigma\in\Gamma_{2m}$ it holds that

[TABLE]

hence,

[TABLE]

Step 3 – Reduction to finite class: Fix $(\mathcal{S},\mathcal{S}^{\prime})$ and consider a random draw of $\sigma\in\Gamma_{2m}$. For each strategy profile $\boldsymbol{h}$, the quantity $\left|\pi_{i}^{\sigma(\mathcal{S})}(\boldsymbol{h})-\pi_{i}^{\sigma(\mathcal{S}^{\prime})}\right|$ is a random variable. Since $\mathcal{W}\cap\mathcal{S}$ represents the number of distinct strategy profiles in the empirical game over $\mathcal{S}$ (see Subsection 3.1), there are at most $\Pi_{\mathcal{W}}(2m)$ such random variables.

[TABLE]

Step 4 – Hoeffding’s inequality: By viewing the previous equation as Rademacher random variables, we have

[TABLE]

Observe that for every $j$ it holds that $r_{j}\left(w_{i}(z_{j},\boldsymbol{h})-w_{i}(z_{j+m},\boldsymbol{h})\right)\in[-1,1]$, and

[TABLE]

holds due to symmetry. By applying Hoeffding’s inequality we obtain

[TABLE]

Finally, by combining Equations (16),(17),(18) and (19) we derive the desired result. ∎

Proof of Theorem 3.5.

The theorem follows immediately by applying the union bound on the inequality obtained in Lemma 3.4 and by substituting $\Pi_{\mathcal{W}}(2m)$ according to Claim 3.3 and Lemma 3.2. ∎

Proof of Lemma 3.7.

The lemma is proven by showing that an induced game has a potential function $\Phi:\mathcal{H}\cap\mathcal{S}\rightarrow\mathbb{R}$, and relying on the results of Monderer and Shapley [20]. Namely, we show a function $\Phi$ such that for every $i,\boldsymbol{h}$ and $h_{i}^{\prime}$ it holds that

[TABLE]

Let $N(z_{j};\boldsymbol{h})$ denote the number of players satisfying the point $z_{j}$ under $\boldsymbol{h}$, namely $N(z_{j};\boldsymbol{h})=\sum_{i=1}^{N}\mathcal{I}(z_{j},h_{i})$. Next, let $\Phi(\boldsymbol{h})\stackrel{{\scriptstyle\text{def}}}{{=}}\frac{1}{m}\sum_{j=1}^{m}\sum_{k=1}^{N(z_{j};\boldsymbol{h})}\frac{1}{k}$, and observe that

[TABLE]

Since the empirical game is finite (i.e., finite number of players with finite strategy space for each player), we conclude that it possesses at least one PNE. ∎

Proof of Lemma 3.9.

Let $N(z_{j};\boldsymbol{h})$ as in the proof of Lemma 3.7. In each iteration of the dynamics it holds that

[TABLE]

Notice that

[TABLE]

Since the potential is bounded by $\ln N+1$ and increases by at least $\epsilon$ per iteration throughout the dynamics, after at most $\frac{\ln N+1}{\epsilon}$ iterations, it will reach its maximum value, thereby obtaining an $\epsilon$-PNE. ∎

Proof of Lemma 3.10.

By Equation (5), we look for $m$ that satisfies

[TABLE]

for given $\epsilon,\delta$; thus

[TABLE]

Next,

Claim B.1** ([25], Section A).**

Let $a\geq 1$ and $b>0$. If $m\geq 4a\log(2a)+2b$, then $m\geq a\log m+b$.

Set $a=\frac{80d}{\epsilon^{2}}$ and $b=\frac{80d\log(2e)}{\epsilon^{2}}+\frac{8}{\epsilon^{2}}\log\frac{4N}{\delta}$. Due to Claim B.1, we know that every $m$ that satisfies

[TABLE]

also satisfies Equation (22). ∎

Proof of Lemma 3.11.

Notice that for every $i,h^{\prime}_{i}$ it holds that

[TABLE]

therefore, if $\pi_{i}(h_{i}^{\prime},\boldsymbol{h}_{-i})-\pi_{i}(\boldsymbol{h})>\epsilon$ then at least one of $\pi_{i}(h_{i}^{\prime},\boldsymbol{h}_{-i})-\pi_{i}^{\mathcal{S}}(h_{i}^{\prime},\boldsymbol{h}_{-i})>\frac{\epsilon}{4}$ or $\pi_{i}^{\mathcal{S}}(\boldsymbol{h})-\pi_{i}(\boldsymbol{h})>\frac{\epsilon}{4}$ must hold. Overall,

[TABLE]

∎

Proof of Theorem 4.1.

The proof of this theorem is identical to the proof of Ben-Porat and Tennenholtz [6, Theorem 2], and hence omitted. ∎

Proof of Claim 5.1.

Denote $\mathcal{S}=\left(x_{1},\dots,x_{m}\right)\in\mathcal{X}^{m}$ and $\boldsymbol{r}\in\mathbb{R}^{m}$ such that $\mathcal{H}_{i}$ pseudo-shatters $\mathcal{S}$ with witness $\boldsymbol{r}$. We prove the claim by showing that we can construct $\mathcal{S}^{\prime}\in\mathcal{Z}^{m}$ that is shattered by $\mathcal{F}_{i}$. For every binary vector $\boldsymbol{b}\in\{-1,1\}^{m}$ there exists $h_{\boldsymbol{b}}$ such that $\operatorname{sign}(h_{b}(x_{j})-r_{j})=b_{j}$ for every $j\in[m]$.

Denote $H=\{h_{\boldsymbol{b}}\in\mathcal{H}_{i}\}_{\boldsymbol{b}\in\{-1,1\}^{m}}$ such that $\left|H\right|=2^{m}$. For every $j$ such that $r_{j}\geq 0$, let

[TABLE]

In addition, for every $j$ such that $r_{j}<0$, let

[TABLE]

and denote

[TABLE]

We now show that $\mathcal{F}_{i}$ shatters $\mathcal{S}^{\prime}$. Fix an arbitrary $\boldsymbol{b}\in\{-1,1\}^{m}$, and observe that in case $r_{j}\geq 0$

[TABLE]

where the last argument holds since $h_{\boldsymbol{b}}(x_{j})\geq y_{j}$. Alternatively, if $r_{j}<0$ we have

[TABLE]

where again the last set of inequalities holds since $h_{\boldsymbol{b}}(x_{j})\leq y_{j}$. In case one of Equations (23) and (24) holds in equality, we can slightly shift $r_{j}$ (as was done in the proof of Claim A.1); hence we assume that these are strict inequalities. The expression in Equation (23) corresponds to $\mathcal{I}(h_{\boldsymbol{b}},(x_{j},y_{j},r_{j}-y_{j}))$, while that of Equation (24) corresponds to $\mathcal{I}(h_{\boldsymbol{b}},(x_{j},y_{j},\left|r_{j}\right|+y_{j}))$.

This analysis applies for every $\boldsymbol{b}$; hence, $\mathcal{F}_{i}$ shatters $\mathcal{S}^{\prime}$ as required. ∎

Proof of Claim 5.3.

We prove the claim by showing that for every $h\in\mathcal{H}_{1}$ and $\epsilon>0$ there exists $h^{\prime}\in\mathcal{H}$ such that $\tilde{\pi}_{2}(h,h^{\prime})\geq\frac{2}{3}-\epsilon$. The assertion will then follow due to symmetry ($\mathcal{H}_{1}=\mathcal{H}_{2}$).

Let $h=(a,b)\in\mathcal{H}_{1}$ denote an arbitrary strategy and fix $\epsilon>0$. We denote by $L,C,R$ the left, center and right squares of $U$ with respect to the horizontal axis, i.e.,

[TABLE]

In addition, let $\mathds{1}_{L}(h)$ indicate whether $h$ crosses $L$, namely

[TABLE]

and similarly for $\mathds{1}_{C}(h),\mathds{1}_{R}(h)$. We proceed by an exhaustive case analysis over the alternatives for $\mathds{1}_{L}(h)+\mathds{1}_{C}(h)+\mathds{1}_{R}(h)$:

• •

If $\mathds{1}_{L}(h)+\mathds{1}_{C}(h)+\mathds{1}_{R}(h)\leq 1$, by taking $h^{\prime}$ to be parallel to $h$ and close enough, player 2 can get at least $\frac{2}{3}$.

• •

Else, if $\mathds{1}_{L}(h)+\mathds{1}_{C}(h)+\mathds{1}_{R}(h)=2$, we have three sub-cases:

• –

In case $\mathds{1}_{L}(h)=\mathds{1}_{R}(h)=1$, denote by $L_{ab}$ ($L_{bl}$) the set of points above (below) $h$ inside $L$, namely

[TABLE]

Similarly, let $R_{ab}$ ($R_{bl}$) denote the set of points above (below) $h$ inside $R$. Let $\lambda(A)$ denote the Lebesgue measure (informally, the area) of a set $A\subset\mathbb{R}^{2}$. Notice that by selecting the appropriate $h^{\prime}$, player 2 can obtain a payoff arbitrarily close to

[TABLE]

where the inequality follows from having $\lambda(L_{bl})+\lambda(R_{ab})+\lambda(L_{ab})+\lambda(R_{bl})=\lambda(L)+\lambda(R)=2$.

• –

In case $\mathds{1}_{L}(h)=\mathds{1}_{C}(h)=1$, let $L_{ab},L_{bl}$ be as in the previous case, and also define $C_{ab},C_{bl}$ in the same manner. Here too, player 2 can obtain a payoff arbitrarily close to

[TABLE]

where the inequality follows from having $\lambda(L_{bl})+\lambda(C_{bl})+\lambda(L_{ab})+\lambda(C_{ab})=\lambda(L)+\lambda(C)=2$.

• –

The case of $\mathds{1}_{R}(h)=\mathds{1}_{C}(h)=1$ is symmetric to the previous case.

• •

Finally, if $\mathds{1}_{L}(h)+\mathds{1}_{C}(h)+\mathds{1}_{R}(h)=3$, then $h=(0,1)$. In this case, by taking $h^{\prime}=(0,1+3\epsilon)$, player 2 obtains precisely $\frac{2}{3}-\epsilon$.

∎

Proof of Lemma 5.4.

Let $\tilde{G}=\langle\tilde{\mathcal{Z}},\tilde{\mathcal{D}},\mathcal{N},\mathcal{H},\tilde{\pi}\rangle$. We define an auxiliary game $G=\langle\mathcal{Z},\mathcal{D},\mathcal{N},\mathcal{H},\pi\rangle$ such that $\mathcal{T}=\{0\}$, $\mathcal{Z}=\tilde{\mathcal{Z}}\times\mathcal{T}$ and $\mathcal{D}$ such that

[TABLE]

We use the tilde notation to describe the elements of $\tilde{G}$, and by non-tilde to denote the elements of $G$.

Recall that $\tilde{G}$ is realizable, and let $h_{i}^{*}\in\mathcal{H}_{i}$ such that $h_{i}^{*}(x)=y$ for every $(x,y)\in\tilde{\mathcal{Z}}$. By definition of $\boldsymbol{h}$, which is a function of the sample $\tilde{\mathcal{S}}$, $\pi_{i}^{\mathcal{S}}(h_{i}^{*},\boldsymbol{h}_{-i})\leq\pi_{i}^{\mathcal{S}}(\boldsymbol{h})$ holds in $G$. By slightly modifying the proof of Lemma 3.11 (the first inequality, as $\boldsymbol{h}$ is a PNE of the empirical game of $G$ and not an $\frac{\epsilon}{2}$-PNE as analyzed there), we know that with probability of at least $1-\delta$ over the selection of $\mathcal{S}$ it holds that

[TABLE]

Using the definitions of $\tilde{w}_{i},w_{i}$ and Equation (25) we obtain that

[TABLE]

where the last inequality holds with probability at least $1-\delta$. As a result, with probability at least $1-\delta$,

[TABLE]

This concludes the proof of the lemma. ∎

Appendix C Additional claims and proofs

Claim C.1**.**

For a given $\boldsymbol{h}$ and $m\geq\frac{2}{\epsilon^{2}}$ it holds that

[TABLE]

Proof of Claim C.1.

Recall Chebyshev’s inequality

[TABLE]

Applying it for our problem, we get

[TABLE]

Notice that $\pi_{i}^{\mathcal{S}^{\prime}}(\boldsymbol{h})$ is the average of independent random variables bounded in the $[0,1]$ segment; hence, by Popoviciu’s inequality on variances we have

[TABLE]

Finally, for $m\geq\frac{2}{\epsilon^{2}}$ it holds that

[TABLE]

∎

Claim C.2**.**

Let $m\geq 15$, $(X_{i})_{i=1}^{m}$ be a sequence of i.i.d. Bernoulli r.v. with $p=\frac{1}{2}$, and let $\bar{X}=\frac{1}{m}\sum_{i=1}^{m}X_{i}$. Then $\Pr\left(\frac{1}{2}<\bar{X}<\frac{3}{4}\right)\geq\frac{1}{4}$ .

Proof of Claim C.2.

By Hoeffding’s inequality we have $\Pr\left(\bar{X}\geq(p+\epsilon)\right)\leq\exp\left(-2\epsilon^{2}m\right)$. Therefore

[TABLE]

∎

Appendix D Details for Subsection 4.2

In all simulations, we selected $m=100$ points. The distribution over $\mathcal{X}\times\mathcal{Y}\times\mathcal{T}$ was the product distribution $\mathcal{D}_{\mathcal{X}}\cdot\mathcal{D}_{\mathcal{Y}}\cdot\mathcal{D}_{\mathcal{T}}$, such that $\mathcal{D}_{\mathcal{X}}$ is $Uni[0,5]$ and $\mathcal{D}_{\mathcal{Y}}$ as appear in Subsection 4.2 with the following parameters:

Linear: For every instance $x$, $y=2x+1+\epsilon$ for $\epsilon\sim Normal(0,1)$. 2. 2.

V-shape: The value of every $x$ is determined by $y=5\left|x-2.5\right|+1+\epsilon$ for $\epsilon\sim Normal(0,1)$. 3. 3.

X-shape: For every instance $x$, the value $y$ is distributed as follows:

[TABLE]

where $\epsilon\sim Normal(0,1)$. 4. 4.

Piecewise: For every $x$, the value $y$ is given by

[TABLE]

where $\epsilon\sim Normal(0,1)$.

In addition, the tolerance level was deterministic, i.e., $\mathcal{D}_{\mathcal{T}}$ was the degenerate deterministic distribution over a value $t$, such that

Linear: High - $t=\frac{3}{2}$ ,Medium - $t=\frac{2}{3}$, Low - $t=\frac{1}{3}$. 2. 2.

V-shape: High - $t=\frac{2}{1}$ ,Medium - $t=\frac{1}{1}$, Low - $t=\frac{1}{2}$. 3. 3.

X-shape: High - $t=\frac{2}{1}$ ,Medium - $t=\frac{1}{1}$, Low - $t=\frac{1}{2}$. 4. 4.

Piecewise: High - $t=\frac{3}{1}$ ,Medium - $t=\frac{3}{2}$, Low - $t=\frac{1}{2}$.