Infinite-Duration Bidding Games

TL;DR

This paper introduces and analyzes infinite-duration bidding games on graphs, establishing the existence of threshold budgets for winning strategies in parity and mean-payoff games, with explicit strategies provided.

Contribution

It is the first to study bidding mode in infinite-duration games, extending the concept of threshold budgets to parity and mean-payoff games and providing explicit optimal strategies.

Findings

Threshold budgets exist for parity and mean-payoff games.

Explicit optimal strategies are constructed for both players.

Connection with random-turn games is extended to infinite-duration settings.

Abstract

Two-player games on graphs are widely studied in formal methods as they model the interaction between a system and its environment. The game is played by moving a token throughout a graph to produce an infinite path. There are several common modes to determine how the players move the token through the graph; e.g., in turn-based games the players alternate turns in moving the token. We study the {\em bidding} mode of moving the token, which, to the best of our knowledge, has never been studied in infinite-duration games. The following bidding rule was previously defined and called Richman bidding. Both players have separate {\em budgets}, which sum up to $1$. In each turn, a bidding takes place: Both players submit bids simultaneously, where a bid is legal if it does not exceed the available budget, and the higher bidder pays his bid to the other player and moves the token. The central question studied in bidding games is a necessary and sufficient initial budget for winning the game: a {\em threshold} budget in a vertex is a value $t \in [0,1]$ such that if Player $1$'s budget exceeds $t$, he can win the game, and if Player $2$'s budget exceeds $1-t$, he can win the game. Threshold budgets were previously shown to exist in every vertex of a reachability game, which have an interesting connection with {\em random-turn} games -- a sub-class of simple stochastic games in which the player who moves is chosen randomly. We show the existence of threshold budgets for a qualitative class of infinite-duration games, namely parity games, and a quantitative class, namely mean-payoff games. The key component of the proof is a quantitative solution to strongly-connected mean-payoff bidding games in which we extend the connection with random-turn games to these games, and construct explicit optimal strategies for both players.

Equations

$$\mbox{Po}(v)=\frac{1}{2}\big{(}\mbox{Po}(v^{+})+\mbox{Po}(v^{-})\big{)}+w(v)-\text{MP}(\text{RT}({\mathcal{G}}))\text{ and }\mbox{St}(v)=\frac{1}{2}\big{(}\mbox{Po}(v^{+})-\mbox{Po}(v^{-})\big{)}$$

$$E(\eta )-G(\eta )+I(\eta )\le \text{mbox}St(v)+\text{mbox}Po(v^{-})+w(v)-\text{mbox}Po(u)=$$

$$=\frac{1}{2}\big{(}\mbox{Po}(v^{+})-\mbox{Po}(v^{-})\big{)}+\mbox{Po}(v^{-})+w(v)-\mbox{Po}(u)=\mbox{Po}(v)-\mbox{Po}(u).$$

$$E(\eta )-G(\eta )+I(\eta )=E({\eta }^{\prime })-G({\eta }^{\prime })+I({\eta }^{\prime })-\text{mbox}St(v)+w(v)\le$$

$$\leq\mbox{Po}(v^{\prime})-\mbox{St}(v)+w(v)-\mbox{Po}(u)\leq\mbox{Po}(v^{+})-\frac{1}{2}\big{(}\mbox{Po}(v^{+})-\mbox{Po}(v^{-})\big{)}+w(v)-\mbox{Po}(u)=\mbox{Po}(v)-\mbox{Po}(u).$$

$$2\text{mbox}{Po}_M-E(\pi )\ge I(\pi )-G(\pi )=N(B-B^{\prime })$$

$$B-\frac{2\text{mbox}{Po}_M-k+k_I}{N}\le B^{\prime }$$

$$w^z(v)=\{\begin{array}{ll}w(v)& \text{ if }w(z)\ge 0\\ z\cdot w(v)& \text{ if }w(z)<0\end{array}$$

$${\tilde{w}}^z(v)=\{\begin{array}{ll}w(v)\cdot z& \text{ if }w(v)<0\\ w(v)\cdot \frac{1}{z}& \text{ if }w(v)\ge 0\end{array}$$

Full text

1 Introduction

Two-player infinite-duration games on graphs are an important class of games as they model the interaction between a system and its environment. Questions about the automatic synthesis of a reactive system from its specification [46] can be reduced to finding a winning strategy for the “system” player in a two-player game. The game is played by placing a token on a vertex in the graph and allowing the players to move it through the graph, thus producing an infinite play. The qualitative winner or quantitative payoff of the game is determined according to the play. There are several common modes to define how the players move the token, which are used to model different types of systems. The most well-studied mode is turn-based, where the vertices are partitioned between the players and the player who controls the vertex on which the token is placed, moves it. Other modes include probabilistic and concurrent moves (see [5]).

We study bidding games in which the mode of moving is “bidding”. Intuitively, in each turn, an auction determines which player moves the token. A concrete bidding rule, which was defined and studied for finite-duration games in [37, 38] is called Richman bidding (named after David Richman). Both players have budgets, and in each turn a bidding takes place: The players simultaneously submit bids, where a bid is legal if it does not exceed the available budget, the higher bidder pays the other player, and moves the token. Ties can occur and one needs to devise a mechanism for resolving them (e.g., giving advantage to Player $1$), but our results do not depend on a specific mechanism.

Bidding arises in many settings that are relevant for several communities within Computer Science, and we list several examples below. In Formal Methods, the players in a two-player game often model concurrent processes. Bidding for moving can model an interaction with a scheduler. The process that wins the bidding gets scheduled and proceeds with its computation. Thus, moving has a cost and processes are interested in moving only when it is critical. Bidding for moving can thus be used to obtain a richer notion of fairness. When and how much to bid can be seen as quantifying the resources that are needed for a system to achieve its objective. Other takes on this problem include reasoning about which input signals need to be read by the system at its different states [21, 3] as well as allowing the system to read chunks of input signals before producing an output signal [29, 28, 34]. Also, our bidding game can model scrip systems that use internal currencies in order to prevent “free riding” [32]; namely, agents who use the resources provided by the system without making their own contribution. Such systems are successfully used in various settings such as databases [50], group decision making [49], resource allocation, and peer-to-peer networks (see [31] and references therein). In Algorithmic Game Theory [44], auction design is a central research topic that is motivated by the abundance of auctions for online advertisements [43]. Repeated bidding is a form of a sequential auction [39], which is used in many settings including online advertising. Infinite-duration bidding games can model ongoing auctions and can be used to devise bidding strategies for objectives like: “In the long run, an advertiser’s ad should show at least half of the time”. In Artificial Intelligence, bidding games have been used to reason about combinatorial negotiations [40].

Recall that “bidding” is a mode of moving and can be studied in combination with any objective. Bidding reachability games were studied in [38, 37]: Player $1$ has a target vertex and an infinite play is winning for him iff it visits the target. The central question that is studied regards a necessary and sufficient budget to guarantee winning, called the threshold budget. Formally, we assume that the budgets add up to $1$. The threshold budget is a function $\textsc{Th}:V\rightarrow[0,1]$ such that if Player $1$’s budget exceeds $\textsc{Th}(v)$ at a vertex $v$, then he has a strategy to win the game from $v$. On the other hand, if Player $2$’s budget exceeds $1-\textsc{Th}(v)$, he can win the game from $v$. We illustrate the bidding model and threshold budgets in the following example.

Example 1**.**

Consider the reachability bidding game that is depicted in Figure 2. Player $1$’s goal is to reach $t$, and Player $2$’s goal is to prevent this from happening. What is a necessary and sufficient initial budget for Player $1$ to win from $v_{0}$? We start with a naive solution by showing that Player $1$ can win if his budget exceeds $0.75$. Suppose that the budgets are $\langle 0.75+\epsilon,0.25-\epsilon\rangle$, for Player $1$ and $2$, respectively, for $\epsilon>0$. In the first turn, Player $1$ bids $0.25$ and wins the bidding since Player $2$ cannot bid above $0.25-\epsilon$. He pays his bid to Player $2$ and moves the token to $v_{2}$. Thus, at the end of the round, the budgets are $\langle 0.5+\epsilon,0.5-\epsilon\rangle$ and the token is placed on $v_{2}$. In the second bidding, Player $1$ bids all his budget, wins the bidding since Player $2$ cannot bid above $0.5-\epsilon$, moves the token to $t$, and wins the game.

While an initial budget of $0.75$ suffices for winning, it is is not necessary for winning. We continue to show that the necessary and sufficient budget in $v_{0}$, i.e., the threshold budget, is $2/3$. That is, we show that for every $\epsilon>0$, Player $1$ can win with a budget of $2/3+\epsilon$, and if his initial budget is $2/3-\epsilon$, he loses since Player $2$ can force the game to $v_{1}$. We show a winning strategy for Player $1$ assuming that the initial budgets are $\langle 2/3+\epsilon,1/3-\epsilon\rangle$. Player $1$’s bid in the first bidding is $1/3$, which he wins since Player $2$ cannot bid beyond $1/3-\epsilon$, and moves the token to $v_{2}$. The new budgets are $\langle 1/3+\epsilon,2/3-\epsilon\rangle$. Now, Player $1$ bids $1/3+\epsilon$. If he wins, he proceeds to $t$ and wins the game. Otherwise, Player $2$ wins the bidding, and moves the token back to $v_{0}$. Since Player $2$ wins the bidding, he must overbid Player $1$’s bid and pay Player $1$ at least $1/3+\epsilon$. In the worst case, the new budgets are $\langle 2/3+2\epsilon,1/3-2\epsilon\rangle$. In other words, we are back to $v_{0}$ only that Player $1$’s budget strictly increases. By continuing in a similar manner, Player $1$ forces his budget to increase by a constant. It will eventually exceed $0.75$ from which he can use the naive solution above to win. The same argument shows that Player $1$ wins with a budget of $1/3+\epsilon$ in $v_{2}$. Showing that Player $2$ wins from $v_{0}$ with $1/3+\epsilon$ and from $v_{2}$ with $2/3+\epsilon$, is dual.

To conclude the example, we note that $\textsc{Th}(v_{1})=1$, which intuitively means that even with all the budget, Player $1$ cannot win from $v_{1}$, and $\textsc{Th}(t)=0$, which intuitively means that even with no budget, Player $1$ wins from $t$. ∎

It is shown in [38, 37] that a threshold budget exists in every vertex of a reachability bidding game. Moreover, it is shown that threshold budgets have the following property: the threshold budget of a vertex $v$ equals $\frac{1}{2}(\textsc{Th}(v^{+})+\textsc{Th}(v^{-}))$, where $v^{+}$ and $v^{-}$ are the successors of $v$ with the maximal and minimal threshold budget, respectively. That is, for every successor $v^{\prime}$ of $v$, we have $\textsc{Th}(v^{-})\leq\textsc{Th}(v^{\prime})\leq\textsc{Th}(v^{+})$. For example, in Example 1 we have $\textsc{Th}(v_{0})=2/3=\frac{1}{2}(1+1/3)=\frac{1}{2}(\textsc{Th}(v_{1})+\textsc{Th}(v_{2}))$.

This property of threshold budgets gives rise to an interesting probabilistic connection. In a random-turn game, instead of bidding, in each turn, we toss a fair coin. If it turns “heads” Player $1$ moves, and if it turns “tails”, Player $2$ moves. For a reachability bidding game ${\mathcal{G}}$, we denote by $\text{RT}({\mathcal{G}})$, the random-turn game that is constructed on top of ${\mathcal{G}}$, which is formally a simple stochastic game [23] (see Figure 3). It is well-known that every vertex $v$ in ${\mathcal{G}}$ has a value in $\text{RT}({\mathcal{G}})$, denoted $val(\text{RT}({\mathcal{G}}),v)$, which is the probability that Player $1$ wins when both players play optimally. The probabilistic connection for reachability bidding games is the following: for every vertex $v$ in ${\mathcal{G}}$, $\textsc{Th}(v)$ in ${\mathcal{G}}$ equals $1-val(\text{RT}({\mathcal{G}}),v)$. Random-turn based games have been extensively studied in their own right since the seminal paper [45].

We introduce and study infinite-duration bidding games with richer qualitative objectives as well as quantitative objectives. Parity games are an important class of qualitative games. For example, the problem of reactive synthesis from LTL specifications is reduced to solving a parity game [46]. The vertices in a parity game are labeled by an index in $\{0,\ldots,d\}$, for some d\in\mbox{I!N}, and an infinite play is winning for Player $1$ iff the parity of the maximal index that is visited infinitely often is odd. We show that parity bidding games are linearly-reducible to reachability bidding games allowing us to obtain all positive results from these games; threshold budgets exist and the problem of computing them is no harder than for reachability bidding games, which is in turn in NP and coNP due to the probabilistic connection. We find this result somewhat surprising since for most other modes of moving, parity games are considerably harder than reachability games. The key component of the proof considers bottom strongly-connected components (BSCCs, for short) in the game graph, i.e., strongly-connected components with no exiting edges. We show that the BSCCs can be easily classified into those that are “winning” for Player $1$ and those that are “losing” for him, where in a winning BSCC, Player $1$ wins with any positive initial budget, and in a losing BSCC, Player $2$ wins with any positive initial budget. We can then construct a reachability bidding game by setting the target of Player $1$ to be the winning BSCCs. Finally, we ask whether Player $1$ can not only win, but win in a prompt manner [35]. In Büchi games, which are a special case of parity games, the goal is to visit an accepting vertex infinitely often. We say that Player $1$ wins in a prompt manner if there is a k\in\mbox{I!N} such that visits to accepting vertices occur within $k$ turns. We show a negative result: under mild assumptions, Player $1$ can never win promptly. That is, with any positive budget, Player $2$ can guarantee arbitrarily long periods with no visits to accepting vertices.

The quantitative games we focus on are mean-payoff games. The vertices of a mean-payoff game are labeled by weights in $\mathbb{Z}$ and an infinite play has a payoff, which is the long-run average of the accumulated weights. The payoff is Player $1$’s cost and Player $2$’s reward, thus we refer to the players in a mean-payoff game as Maximizer (Max, for short) and Minimizer (Min, for short). We adapt threshold budgets to mean-payoff games: we ask what is a necessary and sufficient initial budget to guarantee a payoff of [math]. We show that threshold budgets exist in mean-payoff bidding games and that finding them is again in NP and coNP.

The key component of the proof, which consists of our most technically challenging result, is a quantitative solution for strongly-connected mean-payoff bidding games by showing an extended probabilistic connection for these games. We show that the optimal payoff Min can guarantee in a strongly-connected mean-payoff bidding game ${\mathcal{G}}$ does not depend on his initial budget. More formally, there exists a value c\in\mbox{I!R} such that with every positive initial budget, Min can guarantee a payoff of at most $c$ in ${\mathcal{G}}$, and he cannot do better: for every $\epsilon>0$ and with any positive budget, Max can guarantee a payoff that exceeds $c-\epsilon$ in ${\mathcal{G}}$. Moreover, we show that the optimal payoff $c$ equals the value of the random-turn mean-payoff game $\text{RT}({\mathcal{G}})$. Here, $\text{RT}({\mathcal{G}})$ is a stochastic mean-payoff game and its value is defined as the expected payoff when both players play optimally [41].

We show a constructive proof for the claim above in which we construct optimal bidding strategies for the two players. Intuitively, the strategies that we construct perform a de-randomization; with a deterministic bidding strategy, the players guarantee that the ratio of the time that is spent in each vertex is the same as in a random behavior. We illustrate our construction in the following example. Technically, consider an infinite play $\pi$. The energy of a prefix $\pi^{n}$ of length $n$ of $\pi$, denoted $E(\pi^{n})$, is the sum of the weights that it traverses. The payoff of $\pi$ is $\lim\inf_{n\to\infty}E(\pi^{n})/n$. Note that the definition favors Min. The strategy we construct for Min guarantees that an infinite play $\pi$ either has (1) infinitely many prefixes with $E(\pi^{n})=0$, or (2) the energy is eventually bounded, thus there is N\in\mbox{I!N} such that, after some point $M$, for every n\in\mbox{I!N} with $n>M$, we have $E(\pi^{n})\leq N$. It is not hard to see that this property implies that the payoff of $\pi$ is non-positive. We stress the point that there are two “currencies” in the game: the players’ budgets are “monopoly money” that they do not care about, rather a player’s goal is to optimize the payoff, which arises from the weights that are traversed by the play.

Example 2**.**

Consider the mean-payoff bidding game ${\mathcal{G}}$ that is depicted in Figure 2. The value of the random-turn game that corresponds to ${\mathcal{G}}$ is [math]. Indeed, in $\text{RT}({\mathcal{G}})$, Min always proceeds to $v_{2}$ and Max always proceeds to $v_{1}$. Since the players are selected to move uniformly at random, the game can be seen as a random walk that takes each edge with probability $0.5$ and stays, in the long run, in $v_{1}$ and in $v_{2}$ the same portion of the time. We claim that Min has a deterministic strategy that guarantees a non-positive payoff. It intuitively guarantees that an infinite play stays in $v_{2}$ for at least half the time. Without loss of generality, Max always proceeds to $v_{1}$ upon winning a bidding. Min’s strategy is a tit-for-tat-like strategy, and he always proceeds to $v_{2}$ upon winning a bidding.

The difficulty is in finding the right bids. Min maintains a queue. When the queue is empty, Min bids [math]. If the queue is not empty, Min bids the smallest element in the queue, and removes it upon winning a bidding. If Max wins a bidding with $b$, then Min adds $b$ to the queue. For example, suppose Max bids $\frac{1}{3},\frac{1}{2},\frac{1}{6}$ in the first three biddings. Min’s first bidding is [math], he loses, and adds $\frac{1}{3}$ to the queue. In the second bidding, Min bids the minimal element $\frac{1}{3}$ in the queue, loses again, and adds $\frac{1}{2}$ to the queue. In the third bidding, Min wins with his bid of $\frac{1}{3}$, removes it from the queue, and his bid in the fourth bidding is $\frac{1}{2}$. For simplicity, we assume that Min wins whenever a tie occurs.

We claim that the tit-for-tat strategy guarantees a non-positive mean-payoff value. Intuitively, elements in Min’s queue can be thought of as Max winnings that are not “matched” by a Min win. Thus, if the size of the queue is $k$, the energy is at most $k$ (this is an upper bound since Min could win with [math] bids). In particular, if the queue is empty, the energy is at most [math]. Suppose the minimal element in the queue is $b$. Then, we claim that the size of the queue, and in turn the accumulated energy, is at most $\lceil 1/{b}\rceil$. Indeed, since each bid in the queue represents an “unmatched” Max bid, if the queue size is greater than $\lceil 1/{b}\rceil$, then the sum of Max’s winning bids is more than $1$, which is impossible since he would need to invest more than the total budget. It follows that Min’s strategy guarantees that in an infinite play either (1) the queue empties infinitely often, thus the energy hits [math] infinitely often, or (2) if there is a point after which the queue stays non-empty, then its size is bounded, hence the energy is bounded. By the above, this property implies a non-positive payoff. ∎

As the tit-for-tat strategy above demonstrates, the strategies that we construct carefully match changes in budget with changes in energy. The first step in our construction for general strongly-connected games is to assign an “importance” to each vertex in the game; the more important a vertex is, the higher a player bids in it. Our definition of importance uses the concept of potentials in stochastic games (see [47]), which were initially used in the context of the strategy iteration algorithm [30]. In the second component of the proof, we find a bid by carefully normalizing the importance of a vertex. Normalization is easier in Min’s case because of the asymmetry in the definition of payoff. As demonstrated in the tit-for-tat strategy, Min keeps the energy bounded from above. Max strategy guarantees that the energy is bounded from below, which is more technically challenging to achieve.

Results on other bidding mechanisms

Since the first publication of this work, further results were obtained on infinite-duration bidding games with other bidding mechanisms. A second bidding rule that was first defined in [37] is called poorman bidding: the winner of a bidding, rather than paying his bid to the loser, pays the bid to the “bank”, thus the sum of budgets decreases as the game proceeds. Poorman bidding naturally model settings in which the scheduler accepts payment such as miners in block-chain technology or the auctioneer in ongoing auctions. The mathematical structure of reachability poorman-bidding games is more involved than with Richman bidding. Namely, no probabilistic connection is known and it is unlikely to exist.

Given the probabilistic connection for reachability Richman-bidding games, the probabilistic connection for mean-payoff Richman-bidding games may not be unexpected. The ideas that were developed in the constructions we show here were later used to show a surprising probabilistic connection for mean-payoff poorman-bidding games in [8], which is in fact richer than the one we observe here for mean-payoff Richman-bidding games. Then, to better understand the curious differences between the seemingly similar bidding rules, infinite-duration bidding games with taxman bidding are studied in [10]. Taxman bidding, which was also defined in [37] and studied for reachability games, span the spectrum between Richman and poorman bidding. A probabilistic connection was shown for these games as well. We elaborate on these results in Section 4.3.1.

Further related work on bidding games

Motivated by recreational games, e.g., bidding chess, discrete bidding games are studied in [25], where the granularity of the bids is bounded by dividing the money into chips. The Richman calculus for reachability continuous-bidding games is extended to discrete-bidding in [25]. Unlike in continuous-bidding, ties play a crucial role in discrete-bidding. The question of which tie-breaking mechanism gives rise to determinacy in infinite-duration discrete-bidding games is investigated in [1]. Non-zero-sum two-player games were studied in [40]. They consider a bidding game on a directed acyclic graph. Moving the token through the graph is done by means of bidding. The game ends once the token reaches a sink, and each sink is labeled with a pair of payoffs for the two players that do not necessarily sum up to [math]. They show existence of subgame perfect equilibrium for every initial budget and a polynomial algorithm to compute it.

2 Preliminaries

A graph game is played on a directed graph $G=\langle V,E\rangle$, where $V$ is a finite set of vertices and $E\subseteq V\times V$ is a set of edges. The neighbors of a vertex $v\in V$, denoted $N(v)$, is the set of vertices $\{u\in V:\langle v,u\rangle\in E\}$. We say that $G$ has out-degree $2$ if for every $v\in V$, we have $|N(v)|=2$. A path in $G$ is a finite or infinite sequence of vertices $\eta=v_{1},v_{2},\ldots$ such that for every $i\geq 1$, we have $\langle v_{i},v_{i+1}\rangle\in E$. A strongly-connected component of $G$ is a set of vertices $S$ such that for every $u,v\in S$ there is a path from $u$ to $v$ in $G$. A bottom strongly-connected component (BSCC, for short) is a maximal strongly-connected component $S$ that has no outgoing edges, i.e., there are no edges of the form $\langle v,u\rangle$, where $v\in S$ and $u\notin S$.

Bidding for moving

A graph game is a two-player game, which proceeds by placing a token on a vertex in a graph and letting the two players move it to produce an infinite play. The play gives rise to a path that determines the qualitative winner or quantitative payoff of the game. We refer to the mechanism that determines how the token moves as the mode of moving of the game. For example, the simplest and most well-studied mode of moving is turn-based; the vertices are partitioned between the two players and the player who controls the vertex on which the token is placed, moves it.

We study a different mode of moving, which we call bidding. Both players have budgets, where for convenience, we have $B_{1}+B_{2}=1$. In each turn, a bidding takes place to determine which player moves the token: Both players simultaneously submit bids, where a bid is a real number in $[0,B_{i}]$, for $i\in\{1,2\}$, the player who bids higher pays the other player and moves the token. Note that the sum of budgets always remains $1$. While draws can occur, our results are not affected by the tie-breaking mechanism that is used. To simplify the presentation, we fix the tie-breaking mechanism to always give advantage to Player $1$.

Strategies and plays

A strategy is a recipe for how to play a game. It is a function that, given a finite history of the game, prescribes to a player which action to take, where we define these two notions below. For example, in turn-based games, a strategy takes as input, the sequence of vertices that were visited so far, and it outputs the next vertex to move to. In bidding games, histories and strategies are more involved since they maintain the information about the bids and winners of the bids. Formally, a history in a bidding game is \pi=\langle v_{1},b_{1},i_{1}\rangle,\ldots,\langle v_{k},b_{k},i_{k}\rangle,v_{k+1}\in(V\times\mbox{I!R}\times\{1,2\})^{*}\cdot V, where for $1\leq j\leq k+1$, the token is placed on vertex $v_{j}$ at round $j$, for $1\leq j\leq k$, the winning bid is $b_{j}$ and the winner is Player $i_{j}$. Consider a finite history $\pi$. For $i\in\{1,2\}$, let $W_{i}(\pi)\subseteq\{1,\ldots,k\}$ denote the indices in which Player $i$ is the winner of the bidding in $\pi$. We denote by $B_{i}(\pi)$ Player $i$’s budget following $\pi$. Let $B^{I}_{i}$ be the initial budget of Player $i$. Player $1$’s budget following $\pi$ is $B_{1}(\pi)=B^{I}_{i}-\sum_{j\in W_{1}(\pi)}b_{j}+\sum_{j\in W_{2}(\pi)}b_{j}$, and Player $2$’s budget is defined dually. Given a history $\pi$ that ends in $v$, a strategy for Player $i$ prescribes an action $\langle b,v\rangle$, where $b\leq B_{i}(\pi)$ is a bid that does not exceed the available budget and $v$ is a vertex to move to upon winning, where we require that $v$ is a neighbor of $v_{k+1}$.

An initial vertex $v_{1}$, initial budgets, and two strategies $f_{1}$ and $f_{2}$ for the players determine a unique infinite play for the game, which we denote by $\text{play}(v_{1},f_{1},f_{2})$, and we define its prefixes inductively. Let $\pi^{1}=v_{1}$. Assume that for $n\geq 1$, we have defined the prefix $\pi^{n}=\langle v_{1},b_{1},i_{1}\rangle,\ldots,\langle v_{k},b_{k},i_{n}\rangle,v_{n+1}$, and we define the prefix $\pi^{n+1}$. Let $\langle b^{\prime},v^{\prime}\rangle=f_{1}(\pi^{n})$ and $\langle b^{\prime\prime},v^{\prime\prime}\rangle=f_{2}(\pi^{n})$ be the two actions proposed by the two players’ strategies. Then, if $b^{\prime}\geq b^{\prime\prime}$, Player $1$ wins and we define $\pi^{n+1}=\langle v_{1},b_{2},i_{1}\rangle,\ldots,\langle v_{k+1},b^{\prime},1\rangle,v^{\prime}$. If $b^{\prime\prime}>b^{\prime}$, Player $2$ wins, and we define $\pi^{n+1}=\langle v_{1},b_{2},i_{1}\rangle,\ldots,\langle v_{k+1},b^{\prime\prime},2\rangle,v^{\prime\prime}$. The path that $\text{play}(v_{1},f_{1},f_{2})$ traverses is $\text{path}(\text{play}(v_{1},f_{1},f_{2}))=v_{1},v_{2},\ldots$.

Objectives

An objective $O$ is a set of infinite paths. Player $1$ wins an infinite play $\pi$ iff $\text{path}(\pi)\in O$. We call a strategy $f$ winning for Player $1$ from a vertex $v$ w.r.t. an objective $O$ if for every strategy $g$ of Player $2$ $\text{play}(v,f,g)$ is winning for Player $1$. Winning strategies for Player $2$ are defined dually. We consider the following qualitative objectives:

In reachability games, Player $1$ has a target vertex $t$ and an infinite play is winning iff it visits $t$. We sometimes use a set of vertices $T$ as the target of Player $1$, then Player $1$ wins iff a vertex in $T$ is visited. 2. 2.

In parity games, each vertex is labeled with an index in $\{1,\dots,d\}$. An infinite path is winning for Player $1$ iff the parity of maximal index visited infinitely often is odd. 3. 3.

Mean-payoff games are played on weighted directed graphs, with weights given by a function $w:V\rightarrow\mathbb{Q}$. Consider an infinite path $\eta=v_{1},v_{2},\dots\in V^{\omega}$. For n\in\mbox{I!N}, the prefix of length $n$ of $\eta$ is $\eta^{n}$, and we define its energy to be $E(\eta^{n})=\sum_{i=1}^{n}w(v_{i})$. The payoff of $\eta$ is $\text{payoff}(\eta)=\liminf_{n\rightarrow\infty}E(\eta^{n})/n$. Player $1$ wins $\eta$ iff $\text{payoff}(\eta)\geq 0$.

Mean-payoff games are quantitative games. We think of the payoff as Player $1$’s reward and Player $2$’s cost, thus in mean-payoff games, we refer to Player $1$ as Max and to Player $2$ as Min. We elaborate on the quantitative solution to mean-payoff games in Section 4.

Threshold budgets

The first question that arises in the context of bidding games asks what is the necessary and sufficient initial budget to guarantee an objective. We generalize the definition in [37, 38]:

Definition 3**.**

(Threshold budgets) Consider a bidding game ${\mathcal{G}}$, a vertex $v$, and an objective $O$ for Player $1$. The threshold budget in $v$, denoted $\textsc{Th}(v)$, is a number in $[0,1]$ such that for an initial budget $B_{I}\in[0,1]$ for Player $1$ we have

• •

if $B_{I}>\textsc{Th}(v)$, then Player $1$ has a winning strategy that guarantees $O$ is satisfied, and

• •

if $B_{I}<\textsc{Th}(v)$, then Player $2$ has a winning strategy that violates $O$.

Random-turn games

A stochastic game is played on an arena $\langle V_{1},V_{2},V_{N},\Delta,\Pr\rangle$, where for $i\in\{1,2\}$, $V_{i}$ is a set of vertices that is controlled by Player $i$, $V_{N}$ is a set of probabilistic vertices that is controlled by “Nature”, where all three sets are disjoint and we denote $V=V_{1}\cup V_{2}\cup V_{N}$, $\Delta\subseteq(V_{1}\cup V_{2})\times V$ is a set of deterministic edges, and $\Pr:V_{N}\times V\rightarrow[0,1]$ are probabilistic transitions, i.e., for each $v\in V_{N}$, we have $\sum_{u\in V}\Pr[v,u]=1$. As in turn-based games, whenever the game reaches a vertex in $V_{i}$ that is controlled by Player $i$, for $i\in\{1,2\}$, he choses how the game proceeds, and whenever the game reaches a vertex $v\in V_{N}$, the next vertex is chosen probabilistically according to $\Pr$.

Consider a bidding game ${\mathcal{G}}$ that is played on a graph $\langle V,E\rangle$. The random-turn game that is associated with ${\mathcal{G}}$ is a stochastic game that intuitively simulates the following process. In each turn we throw a fair coin. If it turns “heads”, then Player $1$ moves the token, and Player $2$ moves if the coin turns “tails”. See an example in Figure 3. Formally, we define $\text{RT}({\mathcal{G}})=\langle V_{1},V_{2},V,\Delta,\Pr\rangle$, where we make two additional copies of each vertex in $V$; for $i\in\{1,2\}$, we have $V_{i}=\{v_{i}:v\in V\}$. Nature vertices simulate the coin toss: for $v\in V$, we have $\Pr[v,v_{1}]=\Pr[v,v_{2}]=1/2$. Reaching a vertex $v_{i}\in V_{i}$, for $i\in\{1,2\}$, means that Player $i$ won the coin toss and gets to choose a neighbor $u\in N(v)$ to move the token to, thus we have $\Delta=\{\langle v_{i},u\rangle:\langle v,u\rangle\in E\text{ and }i\in\{1,2\}\}$.

The objective of Player $1$ in $\text{RT}({\mathcal{G}})$ is the same as his objective in ${\mathcal{G}}$. When ${\mathcal{G}}$ is a reachability game, then $\text{RT}({\mathcal{G}})$ is called a simple stochastic game [24] and the target is the same as in ${\mathcal{G}}$. When ${\mathcal{G}}$ is a mean-payoff game, then $\text{RT}({\mathcal{G}})$ is a stochastic mean-payoff game. The weight of $v_{1},v_{2}$, and $v$ all equal the weight of $v$ in ${\mathcal{G}}$.

The following definitions are standard, and we refer the reader to [47] for more details. Two strategies $f_{1}$ and $f_{2}$ for the two players and an initial vertex $v$ give rise to a probability distribution $D(v,f_{1},f_{2})$ over infinite paths that start in $v$.

Definition 4**.**

(Values in stochastic games) Consider a stochastic game ${\mathcal{G}}$. When ${\mathcal{G}}$ is a qualitative game with objective $O$, the value in a vertex $v$ in ${\mathcal{G}}$, denoted $val({\mathcal{G}},v)$, is $\sup_{f_{1}}\inf_{f_{2}}\Pr_{\eta\sim D(v,f_{1},f_{2})}[\eta\in O]$. When ${\mathcal{G}}$ is a mean-payoff game, the value in $v$ is $\sup_{f_{1}}\inf_{f_{2}}\mathbb{E}_{\eta\sim D(v,f_{1},f_{2})}[\text{payoff}(\eta)]$. For the objectives we consider, positional optimal strategies exist.

The existence of positional optimal strategies implies that by letting Player $2$ choose his strategy before Player $1$, i.e., switching the order in the definitions to $\inf_{f_{2}}\sup_{f_{1}}$, we obtain the same value. Moreover, restricting one or both of the players to use only positional strategies does not change the value.

3 Qualitative Bidding Games

We start by surveying the results of [38, 37] on reachability games before moving to study parity bidding games. The model that is studied in [38, 37] uses a slightly different definition of reachability games, which we call double-reachability games: both players have a target, which we denote by $v_{R}$ and $v_{S}$ for “reach” and “safe”, and the game ends once one of the targets is reached. We assume that all vertices apart from $v_{R}$ and $v_{S}$ have at least one path to both $v_{R}$ and $v_{S}$. We later show that reachability bidding games are equivalent to double-reachability bidding games.

Theorem 5**.**

[38, 37]** Consider a double-reachability bidding game ${\mathcal{G}}=\langle V,E,v_{R},v_{S}\rangle$. Then $\textsc{Th}(v_{R})=0$ and $\textsc{Th}(v_{S})=1$, and for all other vertices $v\in V\setminus\{v_{R},v_{S}\}$, we have \textsc{Th}(v)=\frac{1}{2}\big{(}\textsc{Th}(v^{+})+\textsc{Th}(v^{-})\big{)}, where $v^{-},v^{+}\in N(v)$ are such that for every $v^{\prime}\in N(v)$, we have $\textsc{Th}(v^{-})\leq\textsc{Th}(v^{\prime})\leq\textsc{Th}(v^{+})$. Moreover, for every vertex $v\in V$, we have $\textsc{Th}(v)=1-val(\text{RT}({\mathcal{G}}),v)$.

Proof.

We describe the key ideas in the proof for completeness. Consider two optimal memoryless strategies $f_{1}$ and $f_{2}$ in $\text{RT}({\mathcal{G}})$. For $v\in V\setminus\{v_{S},v_{R}\}$, we define $v^{-},v^{+}\in N(v)$ according to these strategies: let $v^{-}=f_{1}(v_{1})$ and $v^{+}=f_{2}(v)$. It is not hard to see that $val(\text{RT}({\mathcal{G}}),v_{S})=0$ and $val(\text{RT}({\mathcal{G}}),v_{R})=1$, and for every $v\in V\setminus\{v_{S},v_{R}\}$, we have $val(\text{RT}({\mathcal{G}}),v)\in(0,1)$ and val(\text{RT}({\mathcal{G}}),v)=\frac{1}{2}\big{(}val(\text{RT}({\mathcal{G}}),v^{+})+val(\text{RT}({\mathcal{G}}),v^{-})\big{)}. We claim that if Player $1$’s budget $B_{I}$ at $v\in V$ exceeds $1-val(\text{RT}({\mathcal{G}}),v)$, then he wins the game. Thus, we show that $\textsc{Th}(v)\leq 1-val(\text{RT}({\mathcal{G}}),v)$. The proof for the other direction is dual.

For $v\in\{v_{S},v_{R}\}$, the claim is trivial. Let $B(v)=1-val(\text{RT}({\mathcal{G}}),v)$ and $\epsilon>0$ be Player $1$’s “surplus”, namely $B_{I}=B(v)+\epsilon$. Intuitively, Player $1$’s strategy ensures that he either wins the game or his surplus increases by a constant. For $u\in V\setminus\{v_{S},v_{R}\}$, let b(u)=\frac{1}{2}\big{(}val(\text{RT}({\mathcal{G}}),u^{-})-val(\text{RT}({\mathcal{G}}),u^{+})\big{)}, which by this theorem, is equivalent to $\frac{1}{2}(\textsc{Th}(v^{+})-\textsc{Th}(v^{-}))$. For example, in the game that is depicted in Figure 2, we have $\textsc{Th}(v_{1})=1$, $\textsc{Th}(v_{2})=1/3$, and $b(v_{0})=(1-1/3)/2=1/3$.

Let $n=|V|$ and, for $1\leq i\leq n$, we define $\epsilon_{i}=\epsilon\cdot 2^{-i}$. Until he loses a bidding, assuming $0\leq j\leq n-1$ turns have passed and the token is placed on a vertex $u\in V$, Player $1$ bids $b(u)+\epsilon_{n-j}$ and proceeds to $u^{-}$ upon winning. Thus, Player $1$’s bid consists of two parts: the major part is $b(u)$ and the minor part is $\epsilon_{n-j}$. We show that no matter the outcome of the bidding, assuming the game continues to $u^{\prime}$, Player $1$’s budget exceeds $B(u^{\prime})$. If Player $1$ wins the bidding, he moves the token to $u^{-}$. His budget exceeds $B(u^{-})$ since $B(u)-b(u)=B(u^{-})$ and $\sum_{1\leq i\leq n}\epsilon_{i}\leq\epsilon$. On the other hand, if Player $2$ wins, in the worst case, he proceeds to $u^{+}$ and pays Player $1$ at least $b(u)+\epsilon_{n-j}$. Then, Player $1$’s budget exceeds $B(u^{+})$ since $B(u)+b(u)=B(u^{+})$ and $\sum_{0\leq\ell<i}\epsilon_{n-\ell}<\epsilon_{n-j}$. Moreover, the difference in the inequality is at least $\epsilon_{n}$. Since $B(v_{S})=1$ and the total budget is $1$, the invariant implies that Player $2$ cannot win the game. It is not hard to show that if Player $1$ wins $n$ biddings, he wins the game. Finally, suppose Player $1$ loses the $i$-th bidding, for $1\leq i\leq n$, then his surplus increases by at least $\epsilon\cdot 2^{-n}$. By repeatedly following this strategy, if Player $1$ does not win the game, his budget will eventually be close enough to $1$, and he can switch to a strategy that wins $n$ biddings in a row and proceeds to $v_{R}$ on the shortest path. ∎

Example 6**.**

Consider the reachability game ${\mathcal{G}}$ that is depicted in Figure 2 and $\text{RT}({\mathcal{G}})$ that is depicted in Figure 3. The optimal strategy for Player $1$ in $\text{RT}({\mathcal{G}})$ proceeds from $v^{1}_{0}$ to $v_{2}$ and from $v^{1}_{2}$ to $t$. The optimal strategy for Player $2$ proceeds from $v^{2}_{2}$ to $v_{0}$ and from $v^{2}_{0}$ to $v_{1}$. We have $1-val(\text{RT}({\mathcal{G}}),v_{1})=1=val(\text{RT}({\mathcal{G}}),t)$, $val(\text{RT}({\mathcal{G}}),v_{0})=\frac{1}{2}(val(\text{RT}({\mathcal{G}}),v_{2})+val(\text{RT}({\mathcal{G}}),v_{1})$, and $val(\text{RT}({\mathcal{G}}),v_{2})=\frac{1}{2}(val(\text{RT}({\mathcal{G}}),v_{0})+val(\text{RT}({\mathcal{G}}),t)$. Note that for every vertex $v$, we have $val(\text{RT}({\mathcal{G}}),v)=1-\textsc{Th}(v)$. For example, $val(\text{RT}({\mathcal{G}}),v_{0})=1/3=1-\textsc{Th}(v_{0})$. ∎

The following proposition makes the equivalence between reachability and double-reachability bidding games precise. Also, unlike Theorem 5, it handles double-reachability games in which only one player has a target, which will be important in our solution to infinite-duration games. Consider a reachability bidding game ${\mathcal{G}}=\langle V,E,t\rangle$. Let $S\subseteq V$ be the set of vertices that have no path to $t$, and let $T\subseteq V$ be the set of vertices that have no path to vertices in $S$. Note that every vertex in $T$ has a path to $t$ but there might be cycles that are contained in $T$ that do not cross $t$. Intuitively, we construct a double-reachability game from ${\mathcal{G}}$ by merging the vertices in $T$ into the target of Player $1$ and merging the vertices in $S$ into the target of Player $2$. Formally, the double-reachability game that corresponds to ${\mathcal{G}}$ is $\text{DR}({\mathcal{G}})=\langle V^{\prime},E^{\prime},v_{R},v_{S}\rangle$, where $V^{\prime}=(V\setminus(S\cup T))\cup\{v_{R},v_{S}\}$, $E^{\prime}=E\cap(V^{\prime}\times V^{\prime})\cup\{\langle v,v_{R}\rangle:\exists u\in T\text{ s.t. }\langle v,u\rangle\in E\}\cup\{\langle v,v_{S}\rangle:\exists u\in S\text{ s.t. }\langle v,u\rangle\in E\}$.

Proposition 7**.**

Consider a reachability bidding game ${\mathcal{G}}=\langle V,E,t\rangle$. Let $S\subseteq V$ be the set of vertices that have no path to $t$, and let $T\subseteq V$ be the set of vertices that have no path to vertices in $S$. For every $v\in S$, we have $\textsc{Th}(v)=1$, for every $v\in T$, we have $\textsc{Th}(v)=0$, and for every $v\in V\setminus(S\cup T)$, we have that $\textsc{Th}(v)$ in ${\mathcal{G}}$ equals $\textsc{Th}(v)$ in the double-reachability game $\text{DR}({\mathcal{G}})$ that corresponds to ${\mathcal{G}}$.

Proof.

The claim that $\textsc{Th}(v)=1$, for every $v\in S$ is trivial. The proof for $v\in T$ is similar to Theorem 5. Suppose Player $1$’s budget at $v$ is $\epsilon>0$. Assuming $|T|=n$, he chooses $\epsilon_{1}>\ldots>\epsilon_{n}$, and, for $0\leq j\leq n-1$, he bids $\epsilon_{n-j}$ in the $j$-th bidding. Upon winning a bidding, he proceeds to a vertex that is closer to $t$ than the current vertex. As in the proof of Theorem 5, if Player $1$ wins $n$ biddings, he wins the game, and if he loses a bidding, his budget increases by at least $\epsilon_{n}$. By repeatedly following this strategy, his budget will eventually suffice for winning $n$ biddings in a row.

The final case is when $v$ is in $V\setminus(S\cup T)$. Suppose Player $1$ starts in ${\mathcal{G}}$ with a budget of $\textsc{Th}(v)+\epsilon$ in $\text{DR}({\mathcal{G}})$. Player $1$ acts as if his budget is $\textsc{Th}(v)+\epsilon/2$ and uses the winning strategy in $\text{DR}({\mathcal{G}})$ to force the game to a vertex in $T$. Then, he uses the strategy above with an initial budget of $\epsilon/2$ to force the game to $t$. ∎

We proceed to study parity bidding games.

Theorem 8**.**

Parity bidding games are linearly reducible to reachability bidding games. Thus threshold budgets exist in parity bidding games.

Proof.

Consider a parity bidding game ${\mathcal{G}}=\langle V,E,p\rangle$ and let $S$ be a BSCC in ${\mathcal{G}}$. We claim that there is $\alpha_{S}\in\{0,1\}$ such that for every $v\in S$, we have $\textsc{Th}(v)=\alpha$. In case of $\alpha=0$, we call $S$ “winning” for Player $1$, and when $\alpha=1$, we call $S$ “losing” for Player $1$. Let $v\in S$ be the vertex with the maximal parity in $S$. We claim that $S$ is winning for Player $1$ iff $p(v)$ is odd. Suppose $p(v)$ is odd, and the other case is dual. We show that Player $1$ can win from $v$ with an initial budget of $\epsilon>0$. Proposition 7 implies that Player $1$ can force the game from any vertex in $S$ to $v$ with any positive initial budget. Indeed, we construct a reachability bidding game ${\mathcal{G}}_{S}$ by restricting ${\mathcal{G}}$ to $S$ and setting the target of Player $1$ to be $v$. Since $S$ is a BSCC, there is no vertex from which there is no path to $v$, thus the proposition implies that the threshold budgets are all [math]. Finally, Player $1$ splits $\epsilon$ into infinitely many pieces $\epsilon_{1},\epsilon_{2},\ldots$, by defining $\epsilon_{i}=\epsilon\cdot 2^{-i}$, for $i\geq 1$. Initially, he plays as if his budget is $\epsilon_{1}$ and forces the game to visit $v$ using the strategy in the reachability game. Once $v$ is visited, he repeats the strategy with an initial budget of $\epsilon_{2}$. He continues similarly forcing infinitely many visits to $v$. Since $v$ is the vertex with maximal parity in $S$ and it is odd, the strategy guarantees that Player $1$ wins.

We now consider vertices in $V$ that are not in a BSCC. Let $W,L\subseteq V$ be the sets of vertices in $V$ that belong to winning and losing BSCCs for Player $1$, respectively. Let $W^{\prime}$ and $L^{\prime}$ be the sets of vertices with no path to vertices in $L$ and $W$, respectively. Note that $W\subseteq W^{\prime}$ and $L\subseteq L^{\prime}$ and as in the above, for every $v\in W^{\prime}$, we have $\textsc{Th}(v)=0$, and for every $v\in L^{\prime}$, we have $\textsc{Th}(v)=1$. We construct a double-reachability bidding game $\text{DR}({\mathcal{G}})$ by setting the target for Player $1$ to be $W^{\prime}$ and the target for Player $2$ to be $L^{\prime}$. A similar argument to the one above shows that for every $v\in V\setminus(W^{\prime}\cup L^{\prime})$, $\textsc{Th}(v)$ in ${\mathcal{G}}$ equals $\textsc{Th}(v)$ in $\text{DR}({\mathcal{G}})$, and we are done. ∎

We adress the computational complexity of finding threshold budgets. We first phrase the problem as a decision problem.

Definition 9**.**

The input to the THRESH-BUDGET problem is a parity bidding game ${\mathcal{G}}$ and a vertex $v$, and the goal is to decide whether $\textsc{Th}(v)\geq 1/2$.

It is stated in [37] that THRESH-BUDGET is in NP and not known to be in P. It follows from Theorems 8 and 5 that THRESH-BUDGET is linearly reducible to the problem of solving a stochastic reachability game, which is known to be in NP and coNP [24]. Thus, we have the following.

Theorem 10**.**

For parity bidding games, THRESH-BUDGET is in NP and coNP.

We conclude this section by studying a stronger notion of winning that is called promptness [35]. Büchi games are a special case of parity games in which the maximal parity index is $1$: vertices with parity $1$ are called accepting and a play is winning for Player $1$ iff it visits an accepting state infinitely often. We show a negative result for Büchi bidding games: intuitively, we show that under mild assumptions Player $1$ cannot win promptly.

Theorem 11**.**

Consider a strongly-connected Büchi bidding game ${\mathcal{G}}=\langle V,E\rangle$ and let $F\subseteq V$ be a set of accepting vertices. If ${\mathcal{G}}$ contains a cycle that does not traverse a vertex in $F$, then for every k\in\mbox{I!N} and every initial positive budget, Player $2$ can force the game not to visit an accepting vertex for at least $k$ turns.

Proof.

Let $C$ be a cycle in ${\mathcal{G}}$ with no accepting state. We construct a reachability game $Cyc({\mathcal{G}},C,k)=\langle V\times\{0,\ldots,k\},E^{\prime},t^{\prime}\rangle$ from ${\mathcal{G}}$ in which we associate Player $2$ in ${\mathcal{G}}$ with Player $1$ in $Cyc({\mathcal{G}},C,k)$ and his goal is to traverse the cycle $C$ $k$ times in a row. Intuitively, the structure of $Cyc({\mathcal{G}},C,k)$ can be thought of as maintaining a counter: when the token is on the vertex $\langle v,i\rangle$, it means that $C$ was traversed for $i$ times. We describe $E^{\prime}$ formally. Let $C=e_{1},\ldots,e_{n}$ be a sequence of edges and for $e\in E$. Let $i\in\{1,\ldots,k\}$ and $e=\langle v,u\rangle\in E$. Suppose $e$ appears in $C$ but it is not the first edge, thus $e\neq e_{1}$. Then, we have $\langle\langle v,i\rangle,\langle u,i\rangle\rangle\in E^{\prime}$, which means that when the game proceeds on an edge in $C$, the counter stays unchanged. When $e=e_{1}$ is the first edge in $C$ and $i<k$, we increment the counter, thus $\langle\langle v,i\rangle,\langle u,i+1\rangle\rangle\in E^{\prime}$. When $e$ is not in $C$, we reset the counter and drop to the first level, thus $\langle\langle v,i\rangle,\langle u,0\rangle\rangle\in E^{\prime}$. Let $v_{0}$ be the first vertex in $C$. Then, the target of Player $2$ is $\langle v_{0},k\rangle$, which means that the cycle is traversed $k$ times in a row.

It is not hard to see that a winning play for Player $1$ in $Cyc({\mathcal{G}},C,k)$ corresponds to traversing the cycle $C$ $k$ times in a row in ${\mathcal{G}}$. Moreover, since ${\mathcal{G}}$ is strongly-connected, the target is reachable from all the vertices in $Cyc({\mathcal{G}},C,k)$. Thus, by Proposition 7, the threshold budgets are [math] in all vertices.

We describe a Player $2$ strategy in ${\mathcal{G}}$ that ensures that there is no bound on the frequencies of visits to accepting states. Suppose Player $2$ starts with a budget of $\epsilon>0$ in ${\mathcal{G}}$. He splits his budget into infinitely many parts $\epsilon_{1},\epsilon_{2},\ldots$. For $i\geq 1$, suppose the token is on $v\in V$. Player $2$ plays according to Player $1$’s winning strategy from $\langle v,0\rangle$ in $Cyc({\mathcal{G}},C,i)$ with an initial budget of $\epsilon_{i}$ to force the game to cycle $C$ $i$ times in a row. Thus, for every $i\geq 1$, there is a sequence of $i\cdot|C|$ with no visit to an accepting state, and we are done. ∎

4 Mean-Payoff Bidding Games

This section consists of our most technically challenging contribution. We show that threshold budgets exist in mean-payoff bidding games and construct optimal strategies for the players. The key component of the proof is a quantitative solution to strongly-connected mean-payoff bidding games. Similar to the proof structure for parity games, the solution allows us to solve general games by first reasoning about the bottom strongly-connected components of the game and then constructing a reachability game for the rest of the vertices.

Consider a strongly-connected mean-payoff bidding game ${\mathcal{G}}$. Recall that a play in a mean-payoff game has a payoff, which is Min’s cost and Max’s reward. Assuming both players start with a positive initial budget, we are intuitively interested in the minimal payoff Min can guarantee assuming his budget is $r\in(0,1)$.111We use $r$ for “ratio” of the total budget as is used in other bidding mechanisms in which the sum of budgets is not constant. See Section 4.3.1. Since ${\mathcal{G}}$ is strongly-connected and the definition of the payoff is prefix independent, Proposition 7 implies that the optimal payoff does not depend on the initial vertex. Thus, it is meaningful to refer to the mean-payoff value of ${\mathcal{G}}$, which we formally define as follows.

Definition 12**.**

(Mean-payoff value) Consider a strongly-connected game ${\mathcal{G}}$ and $r\in(0,1)$. The mean-payoff value of ${\mathcal{G}}$ w.r.t. $r$, denoted $\text{MP}^{r}({\mathcal{G}})$ is a value c\in\mbox{I!R} such that

• •

If Min’s budget is greater than $r$, then he can guarantee that the payoff is at most $c$.

• •

Min cannot do better: for every $\epsilon>0$, if Max’s initial budget is greater than $1-r$, then he can guarantee a payoff of at least $c-\epsilon$.

We justify the asymmetry in the definition by noting that the definition of the payoff of a play uses $\lim\inf$ and thus gives Min an advantage.

The following theorem consists of the main technical contribution of this section. It intuitively states that the initial budget does not matter in strongly-connected mean-payoff bidding games and shows an extended probabilistic connection for these games. In Section 4.3.1, we contrast this property of the Richman-bidding mechanism that we use with the properties of mean-payoff bidding games with other bidding mechanisms. Recall that the mean-payoff value of a vertex in a stochastic mean-payoff game is the expected payoff of the game when both players play optimally. It is not hard to show that since ${\mathcal{G}}$ is strongly-connected, the mean-payoff values of all the vertices in $\text{RT}({\mathcal{G}})$ is the same, thus it is meaningful to refer to the mean-payoff value of $\text{RT}({\mathcal{G}})$, which we denote by $\text{MP}(\text{RT}({\mathcal{G}}))$.

Theorem 13**.**

Consider a strongly-connected mean-payoff bidding game ${\mathcal{G}}$. The mean-payoff value of ${\mathcal{G}}$ exists and does not depend on the initial budget: there exists c\in\mbox{I!R} such that for every $r\in(0,1)$, we have $\text{MP}^{r}({\mathcal{G}})=c$. Moreover, the value of ${\mathcal{G}}$ equals the mean-payoff value of the random-turn mean-payoff game $\text{RT}({\mathcal{G}})$ in which in each turn, the player who chooses a move is selected uniformly at random, thus for every $r\in[0,1]$, we have $\text{MP}^{r}({\mathcal{G}})=\text{MP}(\text{RT}({\mathcal{G}}))$.

The two cases of Theorem 13 are proven separately for Min in Theorem 21 and for Max in Theorem 35 in the following sections. We now describe the theorem’s implications. Recall that Min wins a mean-payoff game if he can guarantee that the payoff is non-positive.

Theorem 14**.**

Threshold budgets exist in mean-payoff bidding games. The THRESH-BUDGET problem for mean-payoff bidding games is in NP and coNP.

Proof.

Consider a general mean-payoff bidding game ${\mathcal{G}}=\langle V,E,w\rangle$. Consider $v\in V$ that belongs to a BSCC $S$ of ${\mathcal{G}}$. Let ${\mathcal{G}}_{S}$ be the game restricted to $S$. Theorem 13 states that if $\text{MP}(\text{RT}({\mathcal{G}}_{S}))\leq 0$, then with every positive initial budget, Min can guarantee a payoff of at most [math]. Thus, the threshold budget in $v$ is [math]. On the other hand, the theorem implies that if $\text{MP}(\text{RT}({\mathcal{G}}_{S}))>0$, Max can guarantee a positive payoff with any positive initial budget, thus the threshold in $v$ is $1$. In the first case, we call $S$ winning for Min, and in the second case, we call $S$ losing for Min. We construct a double-reachability game $\text{DR}({\mathcal{G}})$ in which we associate Min with Player $1$ and set his target to be the set of vertices from which there is no path to losing BSCC, and the target for Max, which we associate with Player $2$, is the set of vertices from which there is no path to a BSCC that is winning for Min. Similarly to the proof of Theorem 8, the threshold budgets in $\text{DR}({\mathcal{G}})$ coincide with the threshold budgets in ${\mathcal{G}}$.

Finally, we show that THRESH-BUDGET is in NP by showing how to verify that $\textsc{Th}(v)\geq 1/2$. For each BSCC $S$ in ${\mathcal{G}}$, we guess positional strategies in the stochastic game $\text{RT}({\mathcal{G}}_{S})$. In addition, we guess two target sets of vertices $T_{1},T_{2}\subseteq V$ and construct the reachability stochastic game $\text{RT}(\text{DR}({\mathcal{G}}))$ using them. Finally, we guess two positional strategies in $\text{RT}(\text{DR}({\mathcal{G}}))$. We first verify that the strategies are optimal in the mean-payoff stochastic games, which can be done in polynomial time. Thus, we obtain the values in all these games. We use the values to verify our guess of the targets $T_{1}$ and $T_{2}$. Namely, we check whether every BSCC $S$ that is winning for Min is contained in $T_{1}$ and that there is no path from a vertex in $T_{1}$ to a BSCC that is winning for Max, and dually for $T_{2}$. Finally, we verify that the positional strategies in the reachability stochastic game are optimal. The solution to $\text{RT}(\text{DR}({\mathcal{G}}))$ gives us the threshold budget in $v$ and we accept if it is at least $1/2$. The size of the witness is polynomial in the input and the verification of the guess can be done in polynomial time. The algorithm above shows that THRESH-BUDGET is in coNP since the only change is to accept when $\textsc{Th}(v)<1/2$. ∎

4.1 An optimal Min strategy in strongly-connected mean-payoff bidding games

In this section we construct an optimal strategy for Min in a strongly-connected mean-payoff bidding game. Since the definition of payoff favors Min, this is technically easier than the construction for an optimal strategy for Max, which we construct in the following section.

Consider a strongly-connected mean-payoff bidding game ${\mathcal{G}}$. In this section, we assume w.l.o.g. that $\text{MP}(\text{RT}({\mathcal{G}}))=0$ as otherwise we can decrease all weights by this value. We construct a bidding strategy for Min that, with any positive initial budget, guarantees that the payoff is non-positive.

Recall that the energy of a finite play is the sum of the weights that it traverses. The following lemma shows that it suffices to construct a Min strategy that keeps the energy bounded from above.

Lemma 15**.**

Consider a mean-payoff bidding game ${\mathcal{G}}$. Suppose that for every positive initial budget $\epsilon>0$ and initial energy k_{I}\in\mbox{I!N}, there is a constant N\in\mbox{I!N} such that Min has a strategy $f_{m}$ that keeps the energy bounded by $N$. That is, for every Max strategy $f_{M}$ and initial vertex $u$, a finite play $\pi=\text{play}(u,f_{m},f_{M})$ either reaches energy [math] or has $E(\pi^{n})\leq N$, for every $1\leq n\leq|\pi|$. Then, Min can guarantee a non-positive payoff in ${\mathcal{G}}$.

Proof.

Suppose Min has a strategy $f_{m}$ as the above, and we describe a Min strategy $f^{\prime}_{m}$ that guarantees a non-positive payoff. Suppose Min’s initial budget is $\epsilon>0$. He splits his budget into infinitely many parts $\epsilon_{1},\epsilon_{2},\ldots$. Initially, Min plays according to $f_{m}$ as if his budget is $\epsilon_{1}$ until an energy of [math] is reached. When energy [math] is reached again, he bids [math] until the energy increases. Once the energy is positive, Min plays according to $f_{m}$ with an initial budget of $\epsilon_{2}$ until an energy of [math] is reached, and so on. Thus, the strategy guarantees that either (1) an energy of [math] is reached infinitely often, or (2) if at some point an energy of [math] is never reached, then the energy stays bounded from above. Recall that the definition of the payoff of an infinite path $\eta=v_{1},v_{2},\ldots$ is $\text{payoff}(\eta)=\lim\inf_{n\to\infty}E(\eta^{n})/n$. Note that an infinite path that satisfies one of the properties (1) or (2) above has a non-negative payoff. ∎

The importance of moving in a vertex. The first component of the strategy construction devises a measure of how “important” it is to move in each vertex in the game. Our definition relies on the concept of potential, which was defined in the context of the strategy improvement algorithm to solve stochastic games [30]. The potential of $v$, denoted $\mbox{Po}(v)$, is a known concept in probabilistic models and its existence is guaranteed [47]. We formalize the notion of the “importance” of moving in a vertex $v$ by defining its strength, which we denote by $\mbox{St}(v)$, and is formally the maximal difference in potentials of the neighbors of $v$.

Definition 16**.**

(Potentials and strengths) Consider two optimal positional strategies $f$ and $g$ in $\text{RT}({\mathcal{G}})$, for Min and Max, respectively. Recall that when constructing $\text{RT}({\mathcal{G}})$, for every vertex $v\in V$, we add two copies $v_{{Min}}$ and $v_{{Max}}$, that are controlled by Min and Max, respectively. For $v\in V$, let $v^{-},v^{+}\in V$ be such that $f(v_{Min})=v^{-}$ and $g(v_{Max})=v^{+}$. The potential of $v$ is a function that satisfies the following and the strength in $v$ is the difference in potentials:

[TABLE]

There are optimal strategies for which $\mbox{Po}(v^{-})\leq\mbox{Po}(v^{\prime})\leq\mbox{Po}(v^{+})$, for every $v^{\prime}\in N(v)$, which can be found for example using the strategy iteration algorithm.

Consider a strongly-connected mean-payoff bidding game ${\mathcal{G}}=\langle V,E,w\rangle$. Consider a finite path $\eta=v_{1},\ldots,v_{n}$ in ${\mathcal{G}}$. We intuitively think of $\eta$ as a play, where for every $1\leq i<n$, the bid of Min in $v_{i}$ is $\mbox{St}(v_{i})$ and he moves to $v_{i}^{-}$ upon winning. Thus, if $v_{i+1}=v_{i}^{-}$, we say that Min won in $v_{i}$, and if $v_{i+1}\neq v_{i}^{-}$, we say that Min lost in $v_{i}$. Let $W(\eta)$ and $L(\eta)$ respectively denote the indices in which Min wins and loses in $\eta$. We call Min wins investments and Min loses gains, where intuitively he invests in increasing the energy and gains budget whenever the energy decreases. Let $G(\eta)$ and $I(\eta)$ be the sum of gains and investments in $\eta$, respectively, thus $G(\eta)=\sum_{i\in L(\eta)}\mbox{St}(v_{i})$ and $I(\eta)=\sum_{i\in W(\eta)}\mbox{St}(v_{i})$. Recall that the energy of $\eta$ is $E(\eta)=\sum_{1\leq i<n}w(v_{i})$. The following lemma connects the strength, potential, and accumulated energy.

Lemma 17**.**

Consider a strongly-connected game ${\mathcal{G}}$ with $\text{MP}(\text{RT}({\mathcal{G}}))=0$, and a finite path $\eta$ in ${\mathcal{G}}$ from $v$ to $u$. Then, $\mbox{Po}(v)-\mbox{Po}(u)\geq E(\eta)-G(\eta)+I(\eta)$.

Proof.

We prove by induction on the length of $\eta$. For $n=1$, the claim is trivial since both sides of the equation are [math]. Suppose the claim is true for paths of length $n$ and we prove for paths of length $n+1$. We distinguish between two cases. In the first case, Min wins in $v$, thus the second vertex in $\eta$ is $v^{-}$. Let $\eta^{\prime}$ be the prefix of $\eta$ starting from $v^{-}$. Note that since Min wins the first bidding, we have $G(\eta)=G(\eta^{\prime})$ and $I(\eta)=\mbox{St}(v)+I(\eta^{\prime})$. Also, we have $E(\eta)=E(\eta^{\prime})+w(v)$. Combining these, we have $E(\eta)-G(\eta)+I(\eta)=E^{\prime}(\eta)+w(v)-G(\eta^{\prime})+I(\eta^{\prime})+\mbox{St}(v)$. By the induction hypothesis, we have $\mbox{Po}(v^{-})-\mbox{Po}(u)\geq E(\eta^{\prime})-G(\eta^{\prime})+I(\eta^{\prime})$. Combining these with the definition of $\mbox{St}(v)$, we have the following.

[TABLE]

[TABLE]

We continue to the second case in which Max wins in $v$ and let $v^{\prime}$ be the second vertex in $\eta$. Recall that we have $\mbox{Po}(v^{+})\geq\mbox{Po}(v^{\prime})$. Dually to the first case, we have $G(\eta)=\mbox{St}(v)+G(\eta^{\prime})$ and $I(\eta)=I(\eta^{\prime})$.

[TABLE]

[TABLE]

∎

Example 18**.**

Consider the game that is depicted in Figure 4. For each vertex $v$, we depict $v^{-}$ by using a bold edge. We illustrate Lemma 17. Consider the path $\eta=v_{3},v_{2},v_{1},v_{1},v_{4}$, which intuitively corresponds to Min winning three biddings and then losing two. For example, we have $v^{-}_{3}=v_{2}$ and $v^{-}_{1}=v_{1}$, thus when losing the bidding in $v_{1}$, Max would proceed to $v_{3}$. The energy is $E(\eta)=-0.5+1-2-2-0.5=-4$ (recall that the last vertex does not contribute to the accumulated energy), the gain is $G(\eta)=\mbox{St}(v_{1})+\mbox{St}(v_{4})=3.5$, the investment is $I(\eta)=\mbox{St}(v_{3})+\mbox{St}(v_{2})+\mbox{St}(v_{1})=5.5$, and the potentials of the two end points are $\mbox{Po}(v_{3})=0$ and $\mbox{Po}(v_{4})=2$. Plugging in the values, we have $\mbox{Po}(v_{3})-\mbox{Po}(v_{4})=0-2=-4-3.5+5.5=E(\eta)-G(\eta)+I(\eta)$. ∎

Normalizing the strengths. The second component in Min’s strategy is a normalization of the strengths, which guarantees that the energy is bounded from above. We develop the intuition in the following example.

Example 19**.**

Consider the mean-payoff bidding game that is depicted in Figure 2. In Example 2, we showed the tit-for-tat Min strategy that bounds the energy from above, which, by Lemma 15, suffices to guarantee a non-positive payoff. We show an alternative construction that generalizes to general strongly-connected games.

Again, Min always proceeds to $v_{2}$ upon winning a bidding and we assume Max proceeds to $v_{1}$ upon winning a bidding. The difficulty is finding the right bids. It is convenient to assume that the initial energy is a positive number k_{I}\in\mbox{I!N} rather than [math]. Suppose that Min starts with a positive initial budget of $B^{I}_{m}>0$. Min chooses an N\in\mbox{I!N} such that $B^{I}_{m}>\frac{k_{I}}{N}$, which is clearly possible since $k_{I}$ is a constant and $B_{I}$ is positive. Min always bids $\frac{1}{N}$ as long as the energy is positive.

We show that the following invariant is maintained: if the energy level reaches 0\leq k\in\mbox{I!N}, Min’s budget is greater than $\frac{k}{N}$. First, our choice of $N$ implies that the invariant holds initially. Second, assuming that the invariant holds before a bidding, we show that it holds after it. Suppose that the energy is $k$ and Min’s budget is $k/N+\epsilon$. If Min wins, the energy decreases by $1$ to $k-1$ and his budget decreases by $1/N$ to $(k-1)/N+\epsilon$. On the other hand, if Max wins the bidding, he bids at least as much as Min, thus Min’s budget increases by at least $1/N$. The energy increases by $1$ to $k+1$ and Min’s budget increases to $(k+1)/N+\epsilon$. The invariant implies that if the energy does not reach [math], then it is bounded by $N$. Indeed, if the energy reaches $k=N$, Min’s budget is $N/N+\epsilon$, which is impossible since the sum of budgets is $1$.∎

We describe the intuition behind Min’s strategy. In the example above, Min’s strategy puts a price of $1/N$ on changing the energy: whenever the energy decreases by $1$, he pays $1/N$, and whenever the energy increases, he gains at least $1/N$. Lemma 17 allows us to generalize this connection between changes in energy and changes in budget. In a vertex $v$, Min bids $\frac{1}{N}\cdot\mbox{St}(v)$ and proceeds to $v^{-}$ upon winning. For example, consider the game that is depicted in Figure 4 and the cycle $\pi=v_{3},v_{2},v_{1},v_{3}$ that results from Min winning the two biddings followed by a Max win. The change in energy is $w(v_{3})+w(v_{2})+w(v_{1})=-0.5+1-2=-1.5$ and Min’s budget decreased by at most \frac{1}{N}\big{(}\mbox{St}(v_{3})+\mbox{St}(v_{2})-\mbox{St}(v_{1})\big{)}=(1.5+2-2)/N=1.5/N. Thus, Min invests at most $c/N$ in a decrease of $c$ units of energy, and he gains at least $c/N$ units of budget when the energy increases by $c$ units. A similar argument as in the lemma above shows an invariant between the energy and budget and in turn, that the energy stays bounded from above.

To formally define Min’s strategy we show how to choose $N$ in general graphs, which requires some book-keeping due to paths that are not cycles. We call Min’s strategy $f_{m}$. Consider a positive initial budget $B\in(0,1]$ for Min and an initial energy k_{I}\in\mbox{I!N}. Let $\mbox{Po}_{M}=\max_{v\in V}|\mbox{Po}(v)|$ and $\mbox{St}_{M}=\max_{v\in V}|\mbox{St}(v)|$. We choose N\in\mbox{I!N} such that $B>\frac{k_{I}+\mbox{St}_{M}+2\mbox{Po}_{M}}{N}$. When the game reaches $v\in V$, Min bids $\mbox{St}(v)/N$ and moves to $v^{-}$ upon winning.

Lemma 20**.**

Consider a Max strategy $f_{M}$, an initial energy k_{I}\in\mbox{I!N}, and let $\pi=\text{play}(f_{m},f_{M})$ be a finite play whose energy stays positive. Thus, for every prefix $\pi^{n}$, for $0\leq n\leq|\pi|$, we have $k_{I}+E(\pi^{n})>0$. Let $k=k_{I}+E(\pi)$ be the energy following $\pi$. Then, Min’s budget following $\pi$ is at least $\frac{k+\mbox{St}_{M}}{N}$.

Proof.

The invariant clearly holds initially. With a slight abuse of notation, let $G(\pi)$ be the sum of “gains” in $\text{path}(\pi)$, namely the sum of strengths in vertices in which Max wins the bidding, and similarly $I(\pi)$ be the “investments” in $\text{path}(\pi)$, namely the sum of strengths in vertices in which Min wins the bidding. Let $B$ be Min’s initial budget and $B^{\prime}$ his budget following $\pi$. Since Min bids $\mbox{St}(v)/N$ in a vertex $v$, we have B^{\prime}=B+\big{(}G(\pi)-I(\pi)\big{)}/N. From Lemma 17, we have $2\mbox{Po}_{M}-E(\pi)\geq I(\pi)-G(\pi)$. By combining with $k=k_{I}+E(\pi)$ and re-arranging, we have

[TABLE]

[TABLE]

Since we define $B>\frac{k_{I}+\mbox{St}_{M}+2\mbox{Po}_{M}}{N}$, we obtain that $B^{\prime}>\frac{k+\mbox{St}_{M}}{N}$, and we are done. ∎

Note that the strategy $f_{m}$ is legal, i.e., Min always has sufficient budget to bid according to $f_{m}$. Indeed, for a choice N\in\mbox{I!N} made by the strategy, the maximal bid in a vertex in ${\mathcal{G}}$ is $\mbox{St}_{M}/N$. Lemma 20 implies that Min has sufficient budget for the bid. Moreover, since Min’s budget cannot exceed $1$, Lemma 20 implies that if the energy does not reach [math], then it is bounded by $N-\mbox{St}_{M}$. Combining with Lemma 15, we obtain the first direction in Theorem 13.

Theorem 21**.**

Let ${\mathcal{G}}$ be a strongly-connected mean-payoff bidding game with $\text{MP}(\text{RT}({\mathcal{G}}))=0$. Then, from every vertex in ${\mathcal{G}}$ and with any positive initial budget, Min can guarantee a non-positive payoff.

4.2 An optimal Max strategy in strongly-connected mean-payoff bidding games

In this section we focus on the more challenging task of constructing an optimal strategy for Max: Given a strongly-connected mean-payoff bidding game ${\mathcal{G}}$ with $\text{MP}(\text{RT}({\mathcal{G}}))>0$, we construct a bidding strategy for Max in ${\mathcal{G}}$ that guarantees a positive payoff.

The following lemma reduces the problem of optimizing the payoff to the problem of bounding the energy from below.

Lemma 22**.**

Assume that for every Max initial budget $\epsilon>0$ in a game ${\mathcal{G}}$ with $\text{MP}(\text{RT}({\mathcal{G}}))>0$, he can keep the energy bounded from below by a constant $N({\mathcal{G}},\epsilon)\in\mathbb{Z}$. Then, Max can guarantee a positive mean-payoff value in ${\mathcal{G}}$.

Proof.

Let ${\mathcal{G}}^{\prime}$ be a mean-payoff bidding game that is obtained from ${\mathcal{G}}$ by decreasing $\text{MP}(\text{RT}({\mathcal{G}}))/2$ from all the weights in ${\mathcal{G}}$. It is not hard to see that $\text{MP}(\text{RT}({\mathcal{G}}^{\prime}))=\text{MP}(\text{RT}({\mathcal{G}}))/2$ and in particular it is positive. Let $\epsilon>0$, and suppose Max plays in ${\mathcal{G}}$ according to a strategy that keeps the energy above $N({\mathcal{G}}^{\prime},\epsilon)$ in ${\mathcal{G}}^{\prime}$. For a finite play $\pi$ in ${\mathcal{G}}$ we have $E(\pi)\geq|\pi|\cdot\text{MP}(\text{RT}({\mathcal{G}}))/2+N({\mathcal{G}}^{\prime},\epsilon)$. Since $N({\mathcal{G}}^{\prime},\epsilon)$ is a constant, its contribution to the payoff vanishes as the length of $\pi$ tends to infinity, thus the payoff is at least $\text{MP}(\text{RT}({\mathcal{G}}))/2$, which is positive. ∎

Bounding the energy from below is more challenging than Min’s goal in the previous section of bounding the energy from above. A first attempt for constructing a Max strategy would be to use a similar strategy as the previous section only with reversed roles: Max’s strategy would guarantee that whenever the energy is k\in\mbox{I!N}, his budget exceeds $k/N$, for some N\in\mbox{I!N}. He would ensure that whenever the energy increases by one unit, his budget decreases by at most $1/N$, and whenever the energy decreases by one unit, his budget increases by at least $1/N$. This attempt fails since Min reacts by allowing Max to win for a while and draw the energy all the way up to $N$, where Max’s budget runs out. When Min has all (or most) of the budget, he can win an arbitrary number of biddings in a row. Thus, he can draw the energy arbitrary low, causing Max to lose since the energy would not be bounded from below. The moral of this attempt is that Max should avoid exhausting his budget. He cannot use a fixed normalization factor of $1/N$. Rather, the normalization factor should decrease as the energy increases. In the next two sections we devise a normalization scheme, first in simpler strongly-connected components and then in general ones.

4.2.1 An optimal Max strategy in recurrent mean-payoff bidding games

A game ${\mathcal{G}}$ is called recurrent, if it is strongly-connected and there is a vertex $u\in V$ such that every cycle in ${\mathcal{G}}$ includes $u$ (see Figure 5). We refer to $u$ as the root of $G$. In this section, we construct an optimal strategy for Max in a recurrent mean-payoff bidding game.

An adapted definition of importance. Recall that Min’s strategy in the previous section matches changes in budget with changes in energy. The first component in Max’s strategy makes this connection asymmetric: we find $z>1$, such that when the energy increases by $c$ units, Max invests at most $c$ units of budget, but when the energy decreases by $c$ units, Max gains at least $z\cdot c$ units of budget.

Consider a recurrent mean-payoff bidding game ${\mathcal{G}}=\langle V,E,w\rangle$ with $\text{MP}(\text{RT}({\mathcal{G}}))>0$. We alter the weights to give advantage to Min. For $z>1$, let ${\mathcal{G}}^{z}=\langle V,E,w^{z}\rangle$, where

[TABLE]

Clearly, $\text{MP}(\text{RT}({\mathcal{G}}))\geq\text{MP}(\text{RT}({\mathcal{G}}^{z}))$. We select $z>1$ such that $\text{MP}(\text{RT}({\mathcal{G}}^{z}))\geq 0$. This is possible since by additively changing all the weights in $\text{RT}({\mathcal{G}})$ by a constant $c$, the value changes by $c$. We select $z$ such that $z\cdot\max_{v\in V}w(v)\leq\text{MP}(\text{RT}({\mathcal{G}}))$.

Consider a finite path $\eta$ in ${\mathcal{G}}$. The following lemma connects the energy of $\eta$ in ${\mathcal{G}}$ with its energy in ${\mathcal{G}}^{z}$. Note that $E(\eta)$ might be negative thus neither claim follows from the other. Let $E^{z}(\eta)$ be the accumulated energy in ${\mathcal{G}}^{z}$.

Lemma 23**.**

Consider a finite path $\eta\in cycles(u)$. Then, $E(\eta)\geq E^{z}(\eta)$ and $zE(\eta)\geq E^{z}(\eta)$.

Proof.

Let $E^{\geq 0}(\eta)$ and $E^{<0}(\eta)$ be the sum of non-negative weights and negative weights in $\eta$, respectively. We have $E(\eta)=E^{\geq 0}(\eta)+E^{<0}(\eta)$ and $E^{z}(\eta)=E^{\geq 0}(\eta)+zE^{<0}(\eta)$. The inequality $E(\eta)\geq E^{z}(\eta)$ is immediate. For the second inequality, we multiply the first equality by $z$ and subtract it from the first to get $E^{z}(\eta)-zE(\eta)=E^{\geq 0}(\eta)-zE^{\geq 0}(\eta)\leq 0$, and we are done. ∎

We adapt Lemma 17 to our setting. We find optimal positional strategies $g_{m}$ and $g_{M}$ for Min and Max, respectively, in the stochastic game $\text{RT}({\mathcal{G}}^{z})$. Using them, we define, for each vertex $v\in V$, vertices $v^{-}$ and $v^{+}$ by setting $v^{-}=g_{m}(v_{m})$ and $v^{+}=g_{M}(v_{M})$. We respectively denote by $\mbox{Po}^{z}$ and $\mbox{St}^{z}$, the potential and strength functions of ${\mathcal{G}}^{z}$. For a finite path $\eta=v_{1},\ldots,v_{n}$, we denote $G^{z}(\eta)=\sum_{v_{i+1}\neq v_{i}^{+}}\mbox{St}^{z}(v_{i})$ and $I^{z}(\eta)=\sum_{v_{i+1}=v_{i}^{+}}\mbox{St}^{z}(v_{i})$. The proof of the following lemma is dual to the proof of Lemma 17.

Lemma 24**.**

For a finite path $\eta$ from $v$ to $u$, we have $\mbox{Po}^{z}(v)-\mbox{Po}^{z}(u)\leq E^{z}(\eta)+G^{z}(\eta)-I^{z}(\eta)$.

Let $pay(\eta)=I^{z}(\eta)-G^{z}(\eta)$. Combining the two lemmas above, we obtain the required asymmetry between “gaining” and “investing”.

Lemma 25**.**

Consider a path $\eta\in cycles(u)$. When $E(\eta)\geq 0$, we have $pay(\eta)\leq E(\eta)$, and when $E(\eta)<0$, we have $-pay(\eta)\geq-z\cdot E(\eta)$.

Example 26**.**

Consider the recurrent mean-payoff bidding game ${\mathcal{G}}$ that is depicted in Figure 5. We have $\text{MP}(\text{RT}({\mathcal{G}}))=0.5$, thus Max can guarantee a positive payoff. We illustrate Lemma 25. The weights of the vertices in ${\mathcal{G}}$ are depicted on top, and, for $z=3/2$, the weights of the negative-weighted vertices are depicted below. Consider the path $\eta=u,v_{1},v_{2},u$. Thus, Max wins the first bidding and loses the second. The bids in vertices with only one outgoing edge are [math]. With this choice of $z$, we have $\text{MP}(\text{RT}({\mathcal{G}}^{z}))=0$, thus we get equality between energy and budget in ${\mathcal{G}}^{z}$. Indeed, the change in energy in ${\mathcal{G}}^{z}$ is $E^{z}(\eta)=w^{z}(u)+w^{z}(v_{1})+w^{z}(v_{2})=-3$. On the other hand, Max’s gain in $\eta$ is $\mbox{St}^{z}(v_{1})=3.5$ and his investment is $\mbox{St}^{z}(u)=0.5$, thus his budget increases by $3.5-0.5=3$. However, the “real” change in energy is the one exhibited in ${\mathcal{G}}$, which is $E(\eta)=w(u)+w(v_{1})+w(v_{2})=-2$. Thus, in a decrease of $2$ units of energy, Max gains $2\cdot z=3$ units of budget rather than only $2$.

The worst case for Max is in paths that traverse only positive or only negative weights. In paths that traverse a mix of weights, the inequality in Lemma 25 is strict. For example, consider the path $u,v_{4},v_{5},u$. The change in energy is $w(u)+w(v_{4})+w(v_{5})=-1$ and the change in budget is $\mbox{St}^{z}(u)+\mbox{St}^{z}(v_{4})=0.5+1.5=2>1\cdot 3/2$.∎

Max’s strategy. When the game reaches a vertex $v$, Max bids $\mbox{St}(v)\cdot\gamma$, where $\gamma$ is a normalization factor that depends on the energy in the last visit to $u$. That is, the normalization changes only after visiting $u$. In order to define $\gamma$, we select N\in\mbox{I!N} and partition the natural numbers into energy blocks of size $N$. Each energy block is associated with its own normalization, which we call the currency of the block. Recall that we chose $z>1$. For n\in\mbox{I!N}, the currency of the $n$-th block is $z^{-n}$. The key idea follows from combining with Lemma 25: investing in the $n$-th block is done in the currency of the $n$-th block while gaining in the $n$-block is in the higher currency of the $(n-1)$-th block.

Example 27**.**

Consider the game ${\mathcal{G}}$ that is depicted in Figure 5, and consider two plays $\pi_{1}=u,v_{1},v_{3},u$ and $\pi_{2}=u,v_{1},v_{2},u$. Suppose the energy in $u$ is in the $3$-rd block, thus the currency is $1.5^{-3}$. In $\pi_{1}$, the energy increases, i.e., we have $E(\pi_{1})=4$, and Max matches his change in budget in the currency of the $3$-rd block, i.e., Max invests $(0.5+3.5)\cdot 1.5^{-3}=4\cdot 1.5^{-3}$. On the other hand, in $\pi_{2}$, we have a decrease of energy, i.e., we have $E(\pi_{2})=-2$, and Max gains $(-0.5+3.5)\cdot 1.5^{-3}=2\cdot 1.5^{-2}$, thus we have a connection between changes in energy and budget, only in the higher currency of the $2$-nd block.∎

We choose N\in\mbox{I!N} as follows. Let $cycles(u)$ be the set of paths that are simple cycles from $u$ to itself. A crucial advantage of recurrent games is that all cycles pass through $u$. Our definition relies on the maximal energy of such a cycle, which we denote by $E_{M}=\max_{\eta\in cycles(u)}|E(\eta)|$. We choose N\in\mbox{I!N} such that $N\geq(\mbox{St}^{z}_{M}+3E_{M})/(1-z^{-1})$, where $\mbox{St}^{z}_{M}$ is the maximal strength of a vertex in ${\mathcal{G}}^{z}$. For $n\geq 1$, we refer to the $n$-th block as $N_{n}$, and we have $N_{n}=\{N(n-1),N(n-1)+1,\ldots,Nn-1\}$. We use $\beta^{\downarrow}_{n}$ and $\beta^{\uparrow}_{n}$ to mark the upper and lower boundaries of $N_{n}$, respectively. We use a $N_{\geq n}$ to denote the set $\{N_{n},N_{n+1},\ldots\}$. Consider a finite play $\pi$ that ends in $u$ and let $visit_{u}(\pi)$ be the set of indices in which $\pi$ visits $u$. Let k_{I}\in\mbox{I!N} be an initial energy. We say that $\pi$ visits $N_{n}$ if $k_{I}+E(\pi)\in N_{n}$. We say that $\pi$ stays in $N_{n}$ starting from an index $1\leq i\leq|\pi|$ if for all $j\in visit_{u}(\pi)$ such that $j\geq i$, we have $k_{I}+E(\pi_{1},\ldots,\pi_{j})\in N_{n}$.

We are ready to describe Max’s strategy, which we denote by $f_{M}$. Max chooses k_{I}\in\mbox{I!N} and plays as if the initial energy in $k_{I}$. With the right choice of $k_{I}$, his strategy will keep the energy non-negative. In turn, assuming that the real initial energy is [math], we obtain that the energy stays above $-k_{I}$. Suppose the game reaches a vertex $v$ and the energy in the last visit to $u$ was in $N_{n}$, for $n\geq 1$. Then, Max bids $z^{-n}\cdot\mbox{St}^{z}(v)$ and proceeds to $v^{+}$ upon winning. Consider an initial Max budget $B^{I}_{M}>0$. We choose an initial energy k_{I}\in\mbox{I!N} with which $f_{M}$ guarantees that energy level [math] is never reached. Recall the intuition that increasing the energy by a unit requires an investment of a unit of budget in the right currency. Thus, increasing the energy from the lower boundary $\beta_{n}^{\downarrow}$ of $N_{n}$ to its upper boundary $\beta_{n}^{\uparrow}$, costs $N\cdot z^{-n}$. We define $cost(N_{n})=N\cdot z^{-n}$ and $cost(N_{\geq n})=\sum_{i=n}^{\infty}cost(N_{n})$. A first attempt for the definition of $k_{I}$ would be $\beta_{n}^{\downarrow}$ such that $B^{I}_{M}>cost(N_{\geq n})$, which intuitively means that Max’s initial budget would never run out even if he always wins. This is almost correct. We need some wiggle room to allow for changes in the currency. Also, note that drawing the energy to [math] from $\beta_{n}^{\downarrow}$ would cost Min a total of $\sum_{i=1}^{n}cost(N_{i})$. We choose $k_{I}$ so that this cost is greater than $1$, thus we ensure that the energy never reaches [math].

Definition 28**.**

Let $k_{I}$ be $\beta_{n}^{\downarrow}$ such that $B^{I}_{M}>wiggle\cdot z^{-(n-1)}+cost(N_{\geq n})$, where $wiggle=2E_{M}+\mbox{St}^{z}_{M}$, and $\sum_{i=1}^{n}cost(N_{i})>1$.

Correctness. We prove an invariant on Max’s budget throughout the game, which will imply that the energy never reaches [math] when it starts from $k_{I}$, and hence the correctness of the strategy.

Consider a Min strategy $f_{m}$, and let $\pi=play(f_{m},f_{M})$ be a finite play. Let $visit_{u}(\pi)=\tau^{1}\cdot\ldots\cdot\tau^{m}$ be a partition of $\pi$ such that for each $1\leq i\leq m$, the $\text{path}(\tau_{i})$ is a cycle-less path that ends in $u$. We define a coarser partition of $\pi$ into sub-plays in which the same currency is used (recall that we change currency at $u$ and when switching between energy blocks). Let $\pi=\pi_{1}\cdot\pi_{2}\cdot\ldots\cdot\pi_{\ell}\cdot\pi_{\ell+1}$, where for each $1\leq i\leq\ell$, we have $\pi_{i}=\tau^{i_{1}}\cdot\ldots\tau^{i_{n_{i}}}$, there is an energy block $N_{n}$ such that the sub-play $\tau^{i_{1}}\cdot\ldots\cdot\tau^{i_{n_{i}-1}}$ stays in $N_{n}$ and the sub-play $\pi_{i}$ visits a neighboring energy block $N_{n-1}$ or $N_{n+1}$. We then call $N_{n}$ the energy block of $\pi_{i}$. We use $e^{i}$ to denote the energy at the end of $\pi^{i}$, thus $e^{i}=k_{I}+E(\pi^{i})$. Let $N_{n}$ be the energy block of $\pi_{i}$. There can be two options; either the energy decreases in $\pi_{i}$, thus the energy before it $e^{i-1}$ is in $N_{n+1}$ and the energy after it $e^{i}$ is in $N_{n}$, or it increases, thus $e^{i-1}\in N_{n-1}$ and $e^{i}\in N_{n}$. We then call $\pi^{i}$ decreasing and increasing, respectively.

Recall that $\beta^{\uparrow}_{n}$ and $\beta^{\downarrow}_{n}$ are the upper and lower boundaries of the energy block $N_{n}$. Further recall that $E_{M}$ is the largest energy of a cycle in ${\mathcal{G}}$. Thus, whenever the energy enters $N_{n}$ it is within $E_{M}$ of the boundary (see Figure 6). In the case that $\pi^{i}$ is decreasing, the energy at the end of $\pi^{i}$ is $e^{i}\geq\beta^{\uparrow}_{n}-E_{M}$ and in the case it is increasing, we have $e^{i}\leq\beta^{\downarrow}_{n}+E_{M}$. Let $\ell_{0}=0$, and for $i\geq 1$, let $\ell_{i}=(\beta^{\downarrow}_{n+1}-E_{M})-e^{i}$ in the first case and $\ell_{i}=(\beta^{\downarrow}_{n}+E_{M})-e^{i}$ in the second case. Note that $\ell_{i}\in\{0,\ldots,2E_{M}\}$. We prove the following invariant on Max’s budget when changing between energy blocks.

Lemma 29**.**

For every $i\geq 0$, suppose $\pi^{i}$ ends in $N_{n}$. The budget of Max at the end of $\pi^{i}$ is at least $(wiggle+\ell_{i})\cdot z^{-(\hat{n}-1)}+cost(N_{\geq\hat{n}})$, where $\hat{n}=n+1$ if $\pi^{i}$ is decreasing and $\hat{n}=n$ if $\pi^{i}$ is increasing.

Proof.

The proof is by induction. The base case follows from our choice of initial energy. For $i\geq 1$, assume the claim holds for $\pi^{i-1}$ and we prove for $\pi^{i}$. There are four cases for the energy changes in $\pi_{i}$, which we depict in Figure 6. Recall that $E_{M}$ is maximal energy of a simple cycle from $u$ and that we switch currencies at $u$. Thus, whenever we switch currency it means that the play visits a new energy block, and the first location in the block is within $E_{M}$ of the boundary.

Intuitively, Case $1$ is the simplest and follows from matching energy and budget in $N_{n}$. In Cases $3$ and $4$ Max invests and gains in the “wrong” currency. For example, in Case $3$, if investing and gaining was in the same currency, Max would have gained in the currency of $N_{n}$ instead of the higher currency of $N_{n-1}$. Finally, in Case $2$, we again use this mismatch to ensure that the gain “covers” the cost of $N_{n}$ and in addition there is a “surplus” that covers the required wiggle room.

Let $e^{i-1}$ be the energy at the end of $\pi^{i-1}$. Consider Cases $1$, $3$, and $4$ in the figure. We prove the first of these case and the others are similar. In Case $3$, we have $e^{i-1}\in N_{n+1}$, and $\pi_{i}$ decreases into $N_{n}$ and $e^{i}$ is near $\beta^{\uparrow}_{n}$. Thus, we have $\ell_{i-1}=(\beta^{\downarrow}_{n+1}+w_{M})-e^{i-1}$ and $\ell_{i}=(\beta^{\downarrow}_{n+1}+w_{M})-e^{i}$. Since we decrease in blocks, we have $\ell_{i-1}<\ell_{i}$ and $E(\pi_{i})=\ell_{i-1}-\ell_{i}$. By Lemma 25, we have $z^{n+1}\cdot pay(\pi_{i})\geq z\cdot(\ell_{i-1}-\ell_{i})$, thus the gain in budget in $\pi_{i}$ is at least $(\ell_{i}-\ell_{i-1})z^{-n}$. The induction hypothesis states that Max’s budget in $\pi^{i-1}$ is at least $(E_{M}+\mbox{St}_{M}+\ell_{i-1})\cdot z^{-n}+\sum_{j=n}^{\infty}Nz^{-j}$, thus his budget after $\pi^{i}$ is at least $(E_{M}+\mbox{St}_{M}+\ell_{i})\cdot z^{-n}+\sum_{j=n}^{\infty}Nz^{-j}$, and we are done. The final case, which is similar to Case $2$ in the figure with a slight difference; the figure depicts energy that crosses $N_{n}$ and we prove for a energy that crosses $N_{n+1}$ and ends in $N_{n}$. That is, the energy at $\pi^{i-1}$ is in $N_{n+1}$ and $e^{i-1}\geq\beta^{\uparrow}_{n+1}-E_{M}$ and $e^{i}\leq\beta^{\downarrow}_{n+1}=\beta^{\uparrow}_{n}$. The decrease in energy is $E(\pi_{i})=(2E_{M}-\ell_{i-1})+(N-2E_{M})+\ell_{i}$, thus by Lemma 25, the increase in budget is $E(\pi_{i})\cdot z^{n-1}$. We chose $N$ such that $(N-2E_{M})\cdot z^{-(n-1)}\geq(E_{M}+\mbox{St}_{M})\cdot z^{-(n-1)}+N\cdot z^{-n}$. The claim follows from combining with the induction hypothesis, and we are done.

∎

It is not hard to show that Lemma 29 implies that $f_{M}$ is legal. That is, consider a finite play $\pi$ that starts immediately after a change in currency. Using Lemma 24, we can prove by induction on the length of $\pi$ that Max has sufficient budget for bidding. The harder case is when $\pi$ decreases, and the proof follows from the fact that $wiggle$ is in the higher currency of the lower block. Combining Lemma 29 with our choice of the initial energy, we get that the energy never reaches [math] as otherwise Min invests a budget of more than $1$. The following theorem follows by combining with Lemma 22.

Theorem 30**.**

In a recurrent mean-payoff bidding game ${\mathcal{G}}$ with $\text{MP}(\text{RT}({\mathcal{G}}))>0$, with any positive initial budget Max has a strategy that guarantees a positive payoff.

4.2.2 An optimal Max strategy in general strongly-connected mean-payoff bidding games

In this section we develop the ideas of the previous section and construct an optimal strategy for Max in general strongly-connected games. Recall that by Lemma 22 it suffices to construct a strategy that guarantees that the energy is bounded from below. The following example shows that naively adapting the strategy from the previous section fails.

Example 31**.**

Consider the strongly-connected mean-payoff bidding game ${\mathcal{G}}$ that is depicted in Figure 7. Note that ${\mathcal{G}}$ is not recurrent. Indeed, the candidates for the root would be $u$ and $v_{1}$ and there are cycles that avoid both of them. We choose $z=2$. With this choice, we have $v_{1}^{+}=v_{1}$. Indeed, $\mbox{Po}^{z}(v_{1})>\mbox{Po}^{z}(v_{3})$. Thus, according to the strategy in the previous section, upon winning a bidding in $v_{1}$, Max chooses the self-loop to stay in $v_{1}$. Since the weight of $v_{1}$ is positive, staying in $v_{1}$ implies an increase of energy, which implies a decrease of budget. Since Max avoids exhausting his budget, the currency must change in $v_{1}$. In other words, Max cannot wait for a visit to $u$ to change the currency. The inability to wait for visits to the root is the challenge of devising a strategy in general strongly-connected games.

A naive solution would be to drop the assumption from the previous section that currency changes occur only at $u$. That is, we change currency upon entering a new energy block no matter what the current vertex is. We illustrate that this attempt fails, implying that a more involved adaptation is needed. The problem is with sinusoidal energy behaviors that occur on the boundary of an energy block. We describe such a play. Consider the cycle $u,v_{1},v_{1},v_{3},v_{5},u$, which intuitively corresponds to Max winning two biddings, then losing two biddings, and we ignore $v_{5}$ since both players bid [math]. In ${\mathcal{G}}^{z}$, we have equality between energy and budget. Indeed, we have $\mbox{St}^{z}(u)+\mbox{St}^{z}(v_{1})-\mbox{St}^{z}(v_{1})-\mbox{St}^{z}(v_{3})=3+0.5-0.5-4=-1=w^{z}(u)+w^{z}(v_{1})+w^{z}(v_{1})+w^{z}(v_{3})+w^{z}(v_{5})$. Note that the “real” energy is the one in ${\mathcal{G}}$ and it is unchanged following this path since $2w(v_{1})+w(v_{5})=0$.

Assume we start from $u$ when the current energy is at the top of the third energy block. Recall that $z=2$, thus the initial currency is $z^{-3}=1/8$. After visiting $v_{1}$ twice, the energy increases and enters the fourth block, thus the currency is updated to $1/16$. Adding the currencies to the calculation above, we get $\mbox{St}^{z}(u)\cdot z^{-3}+\mbox{St}^{z}(v_{1})\cdot z^{-3}-\mbox{St}^{z}(v_{1})\cdot z^{-4}-\mbox{St}^{z}(v_{3})\cdot z^{-4}=3\cdot 1/8+0.5\cdot 1/8-0.5\cdot 1/16-4\cdot 1/16>0$. All in all, Max’s payments are positive, thus his budget decreases, while the energy level stays the same. Min can thus continue with such a strategy until Max’s budget is exhausted. ∎

We develop further the ingredients from the previous sections. Recall that in recurrent games, we split the natural numbers into energy blocks, each block has a currency, where increasing the energy by $c$ units in the $n$-th block costs Max at most $c$ units of budget in the currency of the $n$-th block, and decreasing the energy by $c$ units in the $n$-th block rewards him with at least $c$ units of budget in the higher currency of the $(n-1)$-th block. In general strongly-connected games, we need stronger properties. First, we increase the asymmetry between investing and gaining: while gaining in the $n$-th block is still in the higher currency of the lower $(n-1)$-th block, investing is now in the lower currency of the higher $(n+1)$-th block. Thus, now, in every change to the energy within an energy block, Max registers a profit. The larger the change in energy, the larger the profit. Second, we differentiate between even blocks and odd blocks so that odd blocks serve as “buffers” that ensure that a change in currency only occurs after a significant change in energy.

We formalize this intuition. Consider a strongly-connected mean-payoff bidding game ${\mathcal{G}}=\langle V,E,w\rangle$ having $\text{MP}(\text{RT}({\mathcal{G}}))>0$. For $z>1$, let $\tilde{{\mathcal{G}}}^{z}=\langle V,E,\tilde{w}^{z}\rangle$, where

[TABLE]

As in the previous section, it is not hard to choose $z>1$ such that $\text{MP}(\text{RT}(\tilde{{\mathcal{G}}}^{z}))>0$. Let $\tilde{\mbox{Po}}^{z}$ and $\tilde{\mbox{St}}^{z}$ denote the potentials and strengths in $\tilde{{\mathcal{G}}}^{z}$, and for a finite play we denote by $\tilde{E}^{z}$ be the sum of weights that $\pi$ traverses in $\tilde{{\mathcal{G}}}^{z}$. The proof of the following lemma is similar to Lemma 23.

Lemma 32**.**

Consider a finite play $\pi$. We have $\tilde{E}^{z}(\pi)\leq z\cdot E(\pi)$ and $\tilde{E}^{z}(\pi)\leq\frac{1}{z}\cdot E(\pi)$.

We describe Max’s strategy, which we refer to as $f_{M}$. As in the previous section, Max chooses a k_{I}\in\mbox{I!N} and plays as if that is the initial energy while guaranteeing that the energy never reaches [math]. We specify $k_{I}$ later. When reaching a vertex $v\in V$, Max bids $\tilde{\mbox{St}}^{z}(v)\cdot\gamma$ and moves to $v^{+}$ upon winning, where we define the currency $\gamma\in(0,1)$ next. We partition IN into blocks of \tilde{N}\in\mbox{I!N}, where we choose $\tilde{N}$ later on. We refer to the $n$-th block as $\tilde{N}_{n}=\{\tilde{N}\cdot(n-1),\ldots,\tilde{N}n-1\}$. Unlike the previous section, changes in currency can occur in all vertices and only depend on the energy. The currency in even and odd blocks differs. For n\in\mbox{I!N}, when the energy level reaches an even block $\tilde{N}_{2n}$, the currency is $z^{-n}$. In order to determine the currency in the odd blocks, we take the history of the play into account; the currency matches the currency in the last energy block that was visited before entering $\tilde{N}_{2n+1}$. Thus, if it is $\tilde{N}_{2n}$, then the currency is $z^{-n}$ and if it is $\tilde{N}_{2n+2}$, the currency is $z^{-(n+1)}$. We say that a finite outcome is $\gamma$-consistent when all the bids Max performs in it are made in the same currency $\gamma$. Lemma 24 clearly applies to $\tilde{{\mathcal{G}}}^{z}$. Let the maximal weight of a vertex in ${\mathcal{G}}$ be $w_{M}=\max_{v\in V}|W(v)|$. The following lemma follows from combining Lemma 24 with Lemma 32.

Lemma 33**.**

Consider a $(z^{-n})$-consistent outcome $\pi$ that starts in $v$ and ends in $v^{\prime}$. We have $-pay(\pi)\geq-E(\pi)\cdot z^{-(n-1)}-2w_{M}\cdot z^{-n}$ and $pay(\pi)\leq E(\pi)\cdot z^{-(n+1)}+2w_{M}\cdot z^{-n}$.

Suppose Max is playing according to $f_{M}$ and Min is playing according to some strategy $f_{m}$. Let $\pi=play(f_{m},f_{M})$ be the resulting infinite play. Let $\pi=\pi^{1}\cdot\pi^{2}\cdot\ldots$ be a partition of $\pi$ into maximal finite plays that have a consistent currency. For $i\geq 1$, let e^{i}\in\mbox{I!N} be the energy at the end of $\pi^{i}$, thus $e^{i}=k_{I}+E(\pi^{1}\ldots\pi^{i})$, where $k_{I}$ is an initial energy. Also, let $\beta^{\uparrow}_{n}$ and $\beta^{\downarrow}_{n}$ be respectively, the upper and lower boundaries of the energy block $N_{n}$. Note that $\beta^{\uparrow}_{n}=\beta^{\downarrow}_{n+1}$.

Suppose a sub-play $\pi^{i}$ starts in a vertex $v$ and ends in $u$. We make observations on the budget change during $\pi^{i}$. There are four cases, which are depicted in Figure 8. Note that the currency in Cases $1$ and $3$ is $z^{-n}$ and in Cases $2$ and $4$ it is $z^{-(n+1)}$. The energy change in $\pi^{i}$ in Cases $1$ and $2$ is at least $2\tilde{N}$ and at most $2\tilde{N}+2w_{M}$ and in Cases $3$ and $4$ it is at least $\tilde{N}$ and at most $\tilde{N}+2w_{M}$. We use Lemma 33 to obtain the following:

Lemma 34**.**

The following bounds hold for the change in budget in the four cases depicted in Figure 8.

$pay(\pi^{i})\leq(2\tilde{N}+2w_{M})\cdot z^{-(n+1)}+2w_{M}\cdot z^{-n}$, 2. 2.

$-pay(\pi^{i})\geq 2\tilde{N}\cdot z^{-n}-2w_{M}\cdot z^{-(n+1)}$, 3. 3.

$pay(\pi^{i})\leq(\tilde{N}+2w_{M})\cdot z^{-(n+1)}+2w_{M}\cdot z^{-n}$, and 4. 4.

$-pay(\pi^{i})\geq\tilde{N}\cdot z^{-n}+2w_{M}\cdot z^{-(n+1)}$.

To conclude the construction, given an initial Max budget, we find an initial energy level $k_{I}$ with which Max can guarantee that the energy stays positive. We do this by finding an invariant on his budget at the end points of energy blocks. Recall the intuition that Max’s budget should not run out even when the energy increases arbitrarily. We thus require his initial budget to be sufficient to “purchase” all the energy blocks above the initial energy. For n\in\mbox{I!N}, the cost of the blocks $\tilde{N}_{2n}$ and $\tilde{N}_{2n+1}$ is $\tilde{N}\cdot z^{-n}$.

Recall from the previous section that Max’s budget at the bottom of an energy block $\tilde{N}_{2n}$, needs to include, in addition to the costs of the energy blocks $\tilde{N}_{\geq 2n}$, wiggle room in the currency of the lower block. Going back to Lemma 34, we observe that Case $4$ is the only problematic case. Indeed, in all other cases, the path $\pi^{i}$ crosses an energy block whose cost is given in a currency that is lower than the currency of gaining (when decreasing), or higher than the currency of investing (when increasing). Take for example Case $2$. It crosses both $\tilde{N}_{2n+2}$ and $\tilde{N}_{2n+1}$. The cost of $\tilde{N}_{2n+2}$ is $\tilde{N}\cdot z^{-(n+1)}$ whereas the gain for it is roughly $\tilde{N}\cdot z^{-n}$. The situation in Case $4$ is not that bad. The gain equals the cost of $\tilde{N}_{2n+1}$, i.e., $\tilde{N}\cdot z^{-n}$, up to a constant, i.e., $2w_{M}\cdot z^{-n}$. We add this constant in the invariant, thus we require Max’s budget at $\beta^{\uparrow}_{2n+1}$ to include the costs of the higher blocks, the wiggle room, and a surplus of $2w_{M}\cdot z^{-n}$.

We define the invariant on Max’s budget formally. Recall that $wiggle=2w_{M}+\tilde{\mbox{St}}^{z}_{M}$, where $\tilde{\mbox{St}}^{z}_{M}$ is the maximal bid, and it is used to guarantee that $f_{M}$ is legal in a play that stays in an energy block. We write $Inv(\beta^{\uparrow}_{\ell})$ to refer to the budget that Max has when the currency changes near $\beta^{\uparrow}_{\ell}$, thus within $|w_{M}|$ of $\beta^{\uparrow}_{\ell}$. We have the following.

• •

$Inv(\beta^{\uparrow}_{2n})=wiggle\cdot z^{-n}+z^{-n}\tilde{N}+\sum_{i=n+1}^{\infty}2z^{-i}\tilde{N}$, and

• •

$Inv(\beta^{\uparrow}_{2n+1})=wiggle\cdot z^{-n}+2w_{M}\cdot z^{-(n+1)}+\sum_{i=n+1}^{\infty}2z^{-i}\tilde{N}$.

To conclude the construction, we choose $\tilde{N}$ to be large enough so that the invariant is maintained assuming it is maintained initially. Also, given an initial budget for Max, we choose an initial energy level such that the invariant is initially maintained. Combining with Lemma 22, we obtain the second direction of Theorem 13.

Theorem 35**.**

Consider a strongly-connected mean-payoff bidding game ${\mathcal{G}}$ with $\text{MP}(\text{RT}({\mathcal{G}}))>0$. Then, Max has a strategy that guarantees a positive payoff in ${\mathcal{G}}$.

4.3 Remarks

4.3.1 Results for other bidding mechanisms

We elaborate on further results on infinite-duration bidding games that were obtained since an earlier publication of this paper. The bidding mechanism that we study in this paper is called Richman bidding. Poorman bidding is the same as Richman bidding only that the winner of the bidding pays the “bank” rather than the other player. Taxman bidding span the spectrum between Richman and poorman bidding. It is parameterized by a constant $\tau\in[0,1]$: portion $\tau$ of the winning bid is paid to the other player, and portion $1-\tau$ to the bank. Richman bidding is obtained by setting $\tau=1$ and poorman bidding by setting $\tau=0$. Unlike Richman bidding, in both of these mechanisms, the sum of budgets is not constant throughout the game. The central quantity that is studied is thus the ratio of the players’ budget: suppose that for $i\in\{1,2\}$, Player $i$’s budget is $B_{i}$, then Player $1$’s ratio is $B_{1}/(B_{1}+B_{2})$. Note that Player $1$’s ratio coincides with his budget in Richman bidding. For qualitative games, the central question is the existence of a threshold ratio, which is the straightforward adaptation of the threshold budgets we use (see Definition 3).

Reachability games with poorman and taxman bidding have been studied in [37]. It is shown that while threshold ratios exist in reachability poorman and taxman games, the structure of the game is more complicated and no probabilistic connection is known and it is unlikely to exist: already in the reachability game that is depicted in Figure 2, the threshold ratios with poorman bidding are irrational numbers.

Infinite-duration bidding games with poorman bidding were studied in [8] and with taxman bidding in [10]. Given the probabilistic connection for reachability Richman-bidding games (Theorem 5), the probabilistic connection for mean-payoff Richman-bidding games (Theorem 13) may not be unexpected. On the other hand, since no probabilistic connection is known for reachability poorman-bidding games, we find the following probabilistic connection for mean-payoff poorman- and taxman-bidding games surprising. The ideas that were developed in the constructions in this paper played a key role in the proof of the following theorem.

Theorem 36**.**

[8, 10]** Consider a strongly-connected mean-payoff taxman game ${\mathcal{G}}$ and a constant $\tau\in[0,1]$. The optimal payoff Max can guarantee with an initial ratio $r\in(0,1)$ in ${\mathcal{G}}$ equals the value of the biased random-turn game $\text{RT}^{F(\tau,r)}({\mathcal{G}})$, for $F(\tau,r)=\frac{r+\tau\cdot(1-r)}{1+\tau}$, in which in each turn Max is chosen with probability $F(\tau,r)$ and Min with probability $1-F(\tau,r)$. In particular, for poorman bidding, the optimal payoff in ${\mathcal{G}}$ with initial ratio $r$ equals $\text{MP}(\text{RT}^{r}({\mathcal{G}}))$.

Theorem 36 sheds new light on Theorem 13. Richman bidding is the exception of taxman bidding: For every $\tau<1$, the optimal payoff depends both on the structure of the game and the initial ratio. Only in Richman bidding does the optimal payoff depend only on the structure of the game and not on the initial ratio. For example, recall that in the game that is depicted in Figure 2, with Richman bidding, Min can guarantee a non-positive payoff no matter what positive initial budget he starts with (using the tit-for-tat strategy for example). With poorman bidding, on the other hand, when Max’s initial budget is $2$ and Min’s initial budget is $1$, Max’s initial ratio is $\frac{2}{3}$, and the optimal payoff that Max can guarantee is $\frac{2}{3}\cdot 1+\frac{1}{3}\cdot(-1)=\frac{1}{3}$. Theorem 36 implies an interesting connection between Richman and poorman bidding: the value in a mean-payoff bidding game with Richman bidding equals the value with poorman bidding and ratio $0.5$.

4.3.2 An existential proof of Theorem 13

We describe an alternative existential proof of Theorem 13 that relies on a combination of the probabilistic connection for reachability bidding games that are played on infinite graphs [37] and results on probabilistic models [14, 15]. The draw-back of this proof is that it does not give any insight on how to construct optimal strategies. That is, given a strongly-connected mean-payoff bidding game ${\mathcal{G}}$, using Theorem 13 and the existential proof, the only knowledge we obtain is the optimal payoff a player can guarantee. There is no hint, however, on how to construct a strategy that achieves this payoff, which, as can be seen in the previous sections, can be a challenging task.

Existential proof of Theorem 13.

Consider a strongly-connected mean-payoff bidding game ${\mathcal{G}}=\langle V,E,w\rangle$, where w:V\rightarrow\mbox{I!N}. The one-counter game222Sometimes called an energy game [13]. that corresponds to ${\mathcal{G}}$, denoted $\text{OCG}({\mathcal{G}})$, is played on the same graph only with a different objective: a counter tracks the energy in an infinite play $\pi$, and $\pi$ is winning for Min iff there exists a finite prefix in which the energy is [math]. That is, Max wins $\pi$ iff the energy stays positive in every finite prefix of $\pi$. A configuration of $\text{OCG}({\mathcal{G}})$ is a pair \langle v,n\rangle\in V\times\mbox{I!N}, which intuitively means that the token is placed on $v$ and the accumulated energy (the counter value) is $n$. Lemmas 15 and 22 can be rephrased to show the following correspondence between winning in the one-counter bidding game $\text{OCG}({\mathcal{G}})$ and guaranteeing an optimal payoff in ${\mathcal{G}}$:

Claim: If the threshold budget in every configuration $\langle v,n\rangle$ in $\text{OCG}({\mathcal{G}})$ is [math], i.e., Min wins with any positive initial budget, then with every positive initial budget, Min guarantees a non-positive payoff in ${\mathcal{G}}$. On the other hand, if for every vertex $v\in V$ and a positive initial budget $B_{M}>0$ of Max there is an initial energy n\in\mbox{I!N} such that $B_{M}>1-\textsc{Th}(\langle v,n\rangle)$ in $\text{OCG}({\mathcal{G}})$, i.e., Max can prevent Min from winning when the game starts from $\langle v,n\rangle$, then Max can guarantee a positive payoff in ${\mathcal{G}}$.

The game $\text{OCG}({\mathcal{G}})$ is a reachability bidding game that is played on an infinite graph. Formally, we have \text{OCG}({\mathcal{G}})=\langle V\times\mbox{I!N},E^{\prime},T\rangle, where $\langle v^{\prime},n^{\prime}\rangle$ is a neighbor of a vertex $\langle v,n\rangle$ iff $\langle v,v^{\prime}\rangle\in E$ and the update to the counter is correct and stays non-negative, i.e., $n^{\prime}=n+w(v)$ if $n^{\prime}\geq 0$ and $n^{\prime}=0$ otherwise, and the target for Min is the set of vertices $V\times\{0\}$. A key property of this game is that even though the graph is infinite, the number of outgoing edges from each vertex is at most $|E|$ and in particular finite. The proof in [38] of the probabilistic connection for reachability bidding games (Theorem 5) extends to reachability games on infinite graphs in which all vertices have a finite out-degree. Thus, we have the following.

Claim: The games $\text{OCG}({\mathcal{G}})$ and $\text{RT}(\text{OCG}({\mathcal{G}}))$ are equivalent: the threshold budget in a configuration \langle v,m\rangle\in V\times\mbox{I!N} in $\text{OCG}({\mathcal{G}})$ equals the value of $\langle v,n\rangle$ in $\text{RT}(\text{OCG}({\mathcal{G}}))$, i.e., the probability of winning under optimal play.

The game $\text{RT}(\text{OCG}({\mathcal{G}}))$ is a stochastic game with a one counter. Such games have been shown to have the following properties.

Claim: [14, 15] When $\text{MP}(\text{RT}({\mathcal{G}}))\leq 0$, the value of every configuration $\langle v,n\rangle$ in $\text{RT}(\text{OCG}({\mathcal{G}}))$ is [math]. When $\text{MP}(\text{RT}({\mathcal{G}}))>0$, for every $v\in V$, the sequence $val(\text{RT}(\text{OCG}({\mathcal{G}})),\langle v,n\rangle)$ tends to [math] as $n$ tends to infinity.

The proof of the theorem follows from combining the three claims. ∎

4.3.3 Strategy complexity

In this section we discuss the memory requirements of the strategies that we construct for mean-payoff bidding games, which we call the complexity of the strategy. The complexity of a strategy is important since strategies are typically used to implement systems, and the complexity of the strategy translates to the complexity of the system. In all three strategies, when the token is placed on a vertex $v$, the strategy always prescribes the same vertex to move to upon winning the bidding, namely $v^{-}$ for Min and $v^{+}$ for Max, and the bid is of the form $\mbox{St}(v)\cdot\gamma$, where $\mbox{St}(v)$ is a constant and $\gamma$ is the normalization factor, which changes as the game proceeds. Thus, a strategy uses memory only for determining the normalization factor.

In Min’s strategy, recall that $\gamma$ is of the form $1/N$, where N\in\mbox{I!N} is chosen immediately after the energy hits [math]. To compute the normalization, Min’s strategy uses two variables that take integer values. One keeps track of the current energy level in order to observe that it hits [math] and that a new $N$ needs to be chosen. The second variable keeps the current choice of $N$.

In Max’s strategy in recurrent games, the normalization, which is called the currency of the energy block, changes in the root vertex of the game depending on the energy level. Max’s strategy again uses two variables that take integer values. The first keeps track of the energy and the second keeps the index of the energy block in the last visit to the root. In a vertex that is not the root, Max computes the normalization by referring to the stored index of the energy block.

Finally, in Max’s strategy in general strongly-connected games, the normalization changes when the energy visits an even energy block. Again, Max’s strategy can be implemented using two variables that keep track of the current energy and the index of the energy block whose currency is currently being used.

5 Discussion and Future Directions

We introduce and study infinite-duration bidding games in which the players bid for the right to move the token. We showed the existence of threshold budgets in parity bidding games by reducing them to reachability bidding games. We also showed the existence of threshold budgets in mean-payoff bidding games. The key to the qualitative solution was a quantitative solution to strongly-connected mean-payoff bidding games: we showed that these games are equivalent to uniform random-turn games in the sense that the optimal payoff a player can guarantee in the bidding game equals the expected payoff in the stochastic game with optimal play. Thus, we show that the initial budgets do not matter in mean-payoff bidding games with the bidding rules we use, namely Richman bidding. That is, the payoff depends only on the structure of the game and not on the initial budgets. As we elaborate in Section 4.3.1, this is not the case with other bidding mechanisms, where the payoff depends both on the structure of the game and the initial budgets.

This work belongs to a line of works that transfer concepts and ideas between the areas of formal verification and algorithmic game theory [44]. Examples of works in the intersection of the two fields include logics for specifying multi-agent systems [4, 20, 42], studies of equilibria in games related to synthesis and repair problems [19, 18, 26, 2], non-zero-sum games in formal verification [12, 16, 22], and applying concepts from formal methods to resource allocation games such as rich specifications [11], efficient reasoning about very large games [6, 36], reasoning about resource interfaces [17], and a dynamic selection of resources [9].

We discuss some directions for future work. We studied the computational complexity of finding threshold budgets, which we formally define as the THRESH-BUDGET problem, and showed that for the objectives we consider, the problem is in NP and coNP using the reduction to random-turn games. We leave open the problem of finding a tighter classification for THRESH-BUDGET. Our result hints that the problem is not NP-hard. A tighter classification would be, optimistically, a polynomial-time algorithm for THRESH-BUDGET, or, pessimistically, showing that THRESH-BUDGET is as hard as solving general simple stochastic games, which is a problem in NP and coNP for which no polynomial-time algorithm is known.

In Section 4.3.2, we discussed one-counter games in which Min wins if the energy hits [math] once in a play. Note that unlike parity and mean-payoff, this objective is not prefix independent. The complexity of THRESH-BUDGET in one-counter games is interesting and is related to recent work on optimizing the probability of reaching a destination in a weighted MDP [27, 48]. For acyclic one-counter bidding games, the problem is PP-hard using a result in [27], and for a single-vertex games the problem is in P using the direct formula of [33]. For general games the problem is open.

Acknowledgments

We thank Petr Novotný and Rasmus Iben-Jensen for helpful discussions and pointers.