On Relevant Equilibria in Reachability Games

TL;DR

This paper investigates the concept of relevant equilibria in multiplayer reachability games, focusing on Pareto optimality and social welfare, and provides complexity results for related decision problems.

Contribution

It introduces notions of relevant equilibria in reachability games and analyzes their computational complexity.

Findings

Existence of Nash and subgame perfect equilibria in reachability games.

Identification of complexity results for decision problems related to relevant equilibria.

Analysis of Pareto optimal and high social welfare equilibria in these games.

Abstract

We study multiplayer reachability games played on a finite directed graph equipped with target sets, one for each player. In those reachability games, it is known that there always exists a Nash equilibrium (NE) and a subgame perfect equilibrium (SPE). But sometimes several equilibria may coexist such that in one equilibrium no player reaches his target set whereas in another one several players reach it. It is thus very natural to identify "relevant" equilibria. In this paper, we consider different notions of relevant equilibria including Pareto optimal equilibria and equilibria with high social welfare. We provide complexity results for various related decision problems.

Tables

Table 1. Table 1: Complexity classes and memory results

Complexity Qual. Reach. Quant. Reach. NE SPE NE SPE Prob. 1 NP-c [10] PSPACE-c[4] NP-c PSPACE-c[6] Prob. 2 NP-c PSPACE-c NP-c PSPACE-c Prob. 3 NP-h/${\Sigma }_2^P$ PSPACE-c NP-h/${\Sigma }_2^P$ PSPACE-c

Memory Qual. Reach. Quant. Reach. NE SPE NE SPE Prob. 1 Poly.[10] Expo.[4] Poly. Expo. Prob. 2 Poly. Expo. Poly. Expo. Prob. 3 Poly. Expo. Poly. Expo.

Equations

$$⟨{\tau }_{\upharpoonright h}{⟩}_v$$

$$⟨({\tau }_i^{\prime },{\tau }_{\upharpoonright h,-i}){⟩}_v$$

$${\mathrm{Val}}_i(v)=\underset{{\sigma }_1\in {\Sigma }_{Min}}{\inf }\underset{{\sigma }_2\in {\Sigma }_{Max}}{\sup }{\mathrm{Cost}}_{\mathrm{Min}}(⟨{\sigma }_1,{\sigma }_2{⟩}_v).$$

$$\underset{{\sigma }_i\in {\Sigma }_{Min}}{\inf }{\mathrm{Cost}}_{\mathrm{Min}}(⟨{\sigma }_i,{\sigma }_{-i}^{\ast }{⟩}_v)={\mathrm{Val}}_i(v)=\underset{{\sigma }_{-i}\in {\Sigma }_{Max}}{\sup }{\mathrm{Cost}}_{\mathrm{Min}}(⟨{\sigma }_i^{\ast },{\sigma }_{-i}{⟩}_v).$$

$${\mathrm{Cost}}_i({\rho }_{\ge k})>{\mathrm{Val}}_i({\rho }_k).$$

$${\mathrm{Cost}}_i({\rho }_{\ge k})={\mathrm{Cost}}_i(⟨{\sigma }_{\upharpoonright h}{⟩}_{{\rho }_k}).$$

$${\mathrm{Val}}_i({\rho }_k)$$

$$\ge {\mathrm{Cost}}_{\mathrm{Min}}(⟨{\sigma }_i^{\ast },{\sigma }_{-i\upharpoonright h}{⟩}_{{\rho }_k})$$

$$={\mathrm{Cost}}_i(⟨{\sigma }_i^{\ast },{\sigma }_{-i\upharpoonright h}{⟩}_{{\rho }_k})$$

$${\mathrm{Cost}}_i(⟨{\sigma }_i^{\ast },{\sigma }_{-i\upharpoonright h}{⟩}_{{\rho }_k})<{\mathrm{Cost}}_i(⟨{\sigma }_{\upharpoonright h}{⟩}_{{\rho }_k}).$$

$${\mathrm{Cost}}_i(h⟨{\sigma }_i^{\ast },{\sigma }_{-i\upharpoonright h}{⟩}_{{\rho }_k})<{\mathrm{Cost}}_i(h⟨{\sigma }_{\upharpoonright h}{⟩}_{{\rho }_k})={\mathrm{Cost}}_i(\rho ).$$

$$P(hv)=\{\begin{array}{ll}\perp & \text{if }P(h)=\perp \text{ and }hv\text{ is a prefix of }\rho ,\\ i& \text{if }P(h)=\perp \text{, }hv\text{ is not a prefix of }\rho \text{ and }h\in {\mathrm{Hist}}_i,\\ P(h)& \text{ otherwise.}\end{array}.$$

$${\sigma }_i(h)=\{\begin{array}{ll}{\tau }_i(h)& \text{if }P(h)=\perp ,\\ {\sigma }_i^{\ast }(h)& \text{if }P(h)=i,\\ {\sigma }_{i,P(h)}^{\ast }(h)& \text{ otherwise}\end{array}.$$

$${\mathrm{Cost}}_i(\tilde{\rho })<{\mathrm{Cost}}_i(\rho ).$$

$${\mathrm{Val}}_i(v)$$

$$\le {\mathrm{Cost}}_{\mathrm{Min}}(⟨{\tilde{\sigma }}_{i\upharpoonright h},{\sigma }_{-i}^{\ast }{⟩}_v)$$

$$={\mathrm{Cost}}_i(⟨{\widetilde{{\sigma }_i}}_{\upharpoonright h},({\sigma }_{j,i}^{\ast }{)}_{j\in \Pi \backslash \{i\}}{⟩}_v).$$

$${\mathrm{Cost}}_i(⟨{\widetilde{{\sigma }_i}}_{\upharpoonright h},({\sigma }_{j,i}^{\ast }{)}_{j\in \Pi \backslash \{i\}}{⟩}_v)\ge {\mathrm{Cost}}_i(⟨{\tau }_{\upharpoonright h}{⟩}_v).$$

$${\mathrm{Cost}}_i(h⟨{\widetilde{{\sigma }_i}}_{\upharpoonright h},({\sigma }_{j,i}^{\ast }{)}_{j\in \Pi \backslash \{i\}}{⟩}_v)\ge {\mathrm{Cost}}_i(h⟨{\tau }_{\upharpoonright h}{⟩}_v).$$

$${\alpha }_u(m,v)=\{\begin{array}{ll}j& \text{if }m=j\in \Pi \text{ or }(m=v_1{v}_2,\text{ with }{v}_1,v_2\in V,v\ne v_2\text{ and }{v}_1\in V_j)\\ {\rho }_t{\rho }_{t+1}& \text{if }m=u{\rho }_t(\text{with }t\in \{0,\dots ,|h\ell |-1\}),u\in V\text{ and }v={\rho }_t\\ {\rho }_n{\rho }_k& \text{ otherwise }(m=u{\rho }_n\text{ and }v={\rho }_n).\end{array}.$$

$${\alpha }_n(m,v)=\{\begin{array}{ll}{\rho }_1& \text{if }m={\rho }_0{\rho }_0\text{ and }v={\rho }_0\\ {\rho }_{t+2}& \text{if }m={\rho }_t{\rho }_{t+1}(\text{with }t\in \{0,\dots ,|h\ell |-2\})\text{ and }v={\rho }_{t+1}\\ {\rho }_k& \text{if }m={\rho }_{n-1}{\rho }_n\text{ and }v={\rho }_n\\ {\sigma }_i^{\ast }(v)& \text{if }m=i\\ {\sigma }_{i,j}^{\ast }(v)& \text{ otherwise}\end{array}.$$

$${\mathrm{Cost}}_i({\rho }_{\ge n}^{\prime })\le {\mathrm{Cost}}_i({\rho }_{\ge \phi (n)})$$

$${\mathrm{Cost}}_i({\rho }_{\ge k}^{\prime })\le {\mathrm{Cost}}_i({\rho }_{\ge \phi (k)}).$$

$${\mathrm{Cost}}_i({\rho }_{\ge \phi (k)})\le \lambda ({\rho }_{\phi (k)})={\lambda }_i({\rho }_k^{\prime })$$

$$\mathcal{I}=\{(0,v_0,I_0)\}$$

$$\text{mbox}with(v,I),(v^{\prime },I^{\prime })\in {\mathrm{Succ}}^{\ast }(v_0,I_0)\text{mbox}andv\in V_i\}.$$

$${\mathrm{Cost}}_i(\rho )\le 1+{\mathrm{Cost}}_i({\rho }_{i,v^{\prime }{I}^{\prime }}).$$

$${\mathrm{Cost}}_i(\rho )\le {\lambda }^{\ast }(v,I)\le 1+{\mathrm{Cost}}_i({\rho }_{i,v^{\prime },I^{\prime }}).$$

$${\mathrm{Cost}}_i(h⟨{\sigma }_{\upharpoonright h}{⟩}_{(v,I)})\le {\mathrm{Cost}}_i(h(v,I)⟨{\sigma }_{\upharpoonright h(v,I)}{⟩}_{(v^{\prime },I^{\prime })}).$$

$${\mathrm{Cost}}_i(h⟨{\sigma }_{\upharpoonright h}{⟩}_{(v,I)})$$

$$=|h(v,I)|+{\mathrm{Cost}}_i(\rho )$$

$$\le |h(v,I)|+1+{\mathrm{Cost}}_i({\rho }_{i,v^{\prime },I^{\prime }})$$

$$={\mathrm{Cost}}_i(h(v,I){\rho }_{i,v^{\prime },I^{\prime }})$$

$$={\mathrm{Cost}}_i(h(v,I)⟨{\sigma }_{\upharpoonright h(v,I)}{⟩}_{v^{\prime },I^{\prime }}).$$

Full text

1 Introduction

Two-player zero-sum games played on graphs are commonly used to model reactive systems where a system interacts with its environment [16]. In such setting the system wants to achieve a goal - to respect a certain property - and the environment acts in an antagonistic way. The underlying game is defined as follows: the two players are the system and the environment, the vertices of the graph are all the possible configurations in which the system can be and an infinite path in this graph depicts a possible sequence of interactions between the system and its environment. In such a game, each player chooses a strategy: it is the way he plays given some information about the game and past actions of the other player. Following a strategy for each player results in a play in the game. Finding how the system can ensure that a given property is satisfied amounts to find, if it exists, a winning strategy for the system in this game. For some situations, this kind of model is too restrictive and a setting with more than two agents such that each of them has his own not necessarily antagonistic objective is more realistic. These games are called multiplayer non zero-sum games. In this setting, the solution concept of winning strategy is not suitable anymore and different notions of equilibria can be studied.

In this paper, we focus on Nash equilibrium (NE) [14]: given a strategy for each player, no player has an incentive to deviate unilaterally from his strategy. We also consider the notion of subgame perfect equilibrium (SPE) well suited for games played on graphs [15]. We study these two notions of equilibria on reachability games. In reachability games, we equip each player with a subset of vertices of the graph game that he wants to reach. We are interested in both the qualitative and quantitative settings. In the qualitative setting, each player only aims at reaching his target set, unlike the quantitative setting where each player wants to reach his target set as soon as possible.

It is well known that both NEs and SPEs exist in both qualitative and quantitative reachability games. But, equilibria such that no player reaches his target set and equilibria such that some players reach it may coexist. This observation has already been made in [19, 18]. In such a situation, one could prefer the second situation to the first one. In this paper, we study different versions of relevant equilibria.

1.0.1 Contributions

For quantitative reachability games, we focus on the following three kinds of relevant equilibria: constrained equilibria, equilibria optimizing social welfare and Pareto optimal equilibria. For constrained equilibria, we aim at minimizing the cost of each player i.e., the number of steps it takes to reach his target set (Problem 1). For equilibria optimizing social welfare, a player does not only want to minimize his own cost, he is also committed to maximizing the social welfare (Problem 2). For Pareto optimal equilibria, we want to decide if there exists an equilibrium such that the tuple of the costs obtained by players following this equilibrium is Pareto optimal in the set of all the possible costs that players can obtain in the game (Problem 3). We consider the decision variant of Problems 1 and 2; and the qualitative adaptations of the three problems.

Our main contributions are the following.(i) We study the complexity of the three decision problems. Our results gathered with previous works are summarized in Table 1.(ii) We characterize a sufficient finite-memory to solve the three decision problems. Our results and others from previous works are given in Table 1.(iii) We identify a subclass of reachability games in which there always exists an SPE where each player reaches his target set.(iv) Given a play, we provide a characterization which guarantees that this play is the outcome of an NE. This characterization is based on the values in the associated two-player zero-sum games called coalitional games.

1.0.2 Related work

There are many results on NEs and SPEs played on graphs, we refer the reader to [9] for a survey and an extended bibliography. We here focus on the results directly related to our contributions.

Regarding Problem 1, for NEs, it is shown NP-complete only in the qualitative setting [10]; for SPEs it is shown PSPACE-complete in both the qualitative and quantitative settings in [4, 6, 5]. Notice that in [19], variants of Problem 1 for games with Streett, parity or co-Büchi winning conditions are shown NP-complete and decidable in polynomial time for Büchi.

Regarding Problem 2, in the setting of games played on matrices, deciding the existence of an NE such that the expected social welfare is at most $k$ is NP-hard [11]. Moreover, in [1] it is shown that deciding the existence of an NE which maximizes the social welfare is undecidable in concurrent games in which a cost profile is associated only with terminal nodes.

Regarding Problem 3, in the setting of zero-sum two-player multidimensional mean-payoff games, the Pareto-curve (the set of maximal thresholds that a player can force) is studied in [2] by giving some properties on the geometry of this set. The autors provide a $\Sigma^{P}_{2}$ algorithm to decide if this set intersects a convex set defined by linear inequations.

Regarding the memory, in [7] it is shown that there always exists an NE with memory at most $|V|+|\Pi|$ in quantitative reachability games, without any constraint on the cost of the NE. It is shown in [18] that, in multiplayer games with $\omega$-regular objectives, there exists an SPE with a given payoff if and only if there exists an SPE with the same payoff but with finite memory. Moreover, in [4] it is claimed that it is sufficient to consider strategies with an exponential memory to solve Problem 1 for SPE in qualitative reachability games.

Finally, we can find several kinds of outcome characterizations for Nash equilibria and variants, e.g., in multiplayer games equipped with prefix-linear cost functions and such that the vertices in coalitional games have a value (summarized in [9]), in multiplayer games with prefix-independent Borel objectives [19], in multiplayer games with classical $\omega$-regular objectives (as reachability) by checking if there exists a play which satisfies an LTL formula [10], in concurrent games [12], etc. Such characterizations are less widespread for subgame perfect equilibria, but one can recover one for quantitative reachability games thanks to a value-iteration procedure [6].

1.0.3 Structure of the paper

Due to the lack of space, we decide to only detail results for quantitative reachability games while results for qualitative reachability games are only summarized in Table 1. In Section 2, we introduce the needed background and define the different studied problems. In Section 3, we identify families of reachability games for which there always exists a relevant equilibrium, for different notions of relevant equilibrium. In Section 4, we provide the main ideas necessary to obtain our complexity results (see Table 1). The detailed proofs for the quantitative reachability setting, together with additional results on qualitative reachability games are provided in the appendices.

2 Preliminaries and studied problems

2.0.1 Arena, game and strategies

An arena is a tuple $\mathcal{A}=(\Pi,V,E,(V_{i})_{i\in\Pi})$ such that: (i) $\Pi$ is a finite set of players; (ii) $V$ is a finite set of vertices; (iii) $E\subseteq V\times V$ is a set of edges such that for all $v\in V$ there exists $v^{\prime}\in V$ such that $(v,v^{\prime})\in E$ and (iv) $(V_{i})_{i\in\Pi}$ is a partition of $V$ between the players.

A play in $\mathcal{A}$ is an infinite sequence of vertices $\rho=\rho_{0}\rho_{1}\ldots$ such that for all $k\in\mathbb{N}$, $(\rho_{k},\rho_{k+1})\in E$. A history is a finite sequence $h=h_{0}h_{1}\ldots h_{k}$ with $k\in\mathbb{N}$ defined similarly. The length $|h|$ of $h$ is the number $k$ of its edges. We denote the set of plays by $\operatorname{Plays}$ and the set of histories by $\operatorname{Hist}$. Moreover, the set $\operatorname{Hist}_{i}$ is the set of histories such that their last vertex $v$ is a vertex of player $i$, i.e. $v\in V_{i}$.

Given a play $\rho\in\operatorname{Plays}$ and $k\in\mathbb{N}$, the prefix $\rho_{0}\rho_{1}\ldots\rho_{k}$ of $\rho$ is denoted by $\rho_{\leq k}$ and its suffix $\rho_{k}\rho_{k+1}\ldots$ by $\rho_{\geq k}$. A play $\rho$ is called a lasso if it is of the form $\rho=h\ell^{\omega}$ with $h\ell\in\operatorname{Hist}$. Notice that $\ell$ is not necessarily a simple cycle. The length of a lasso $h\ell^{\omega}$ is the length of $h\ell$.

A game $\mathcal{G}=(\mathcal{A},(\operatorname{Cost}_{i})_{i\in\Pi})$ is an arena equipped with a cost function profile $(\operatorname{Cost}_{i})_{i\in\Pi}$ such that for all $i\in\Pi$, $\operatorname{Cost}_{i}:\operatorname{Plays}\rightarrow\mathbb{N}\cup\{+\infty\}$ is a cost function which assigns a cost to each play $\rho$ for player $i$. We also say that the play $\rho$ has cost profile $(\operatorname{Cost}_{i}(\rho))_{i\in\Pi}$. Given two cost profiles $c,c^{\prime}\in(\mathbb{N}\cup\{+\infty\})^{|\Pi|}$, we say that $c\leq c^{\prime}$ if and only if for all $i\in\Pi$, $c_{i}\leq c^{\prime}_{i}$.

An initial vertex $v_{0}\in V$ is often fixed, and we call $(\mathcal{G},v_{0})$ an initialized game. A play (resp. a history) of $(\mathcal{G},v_{0})$ is then a play (resp. a history) of $\mathcal{G}$ starting in $v_{0}$. The set of such plays (resp. histories) is denoted by $\operatorname{Plays}(v_{0})$ (resp. $\operatorname{Hist}(v_{0})$). The notation $\operatorname{Hist}_{i}(v_{0})$ is used when these histories end in a vertex $v\in V_{i}$.

Given a game $\mathcal{G}$, a strategy for player $i$ is a function $\sigma_{i}:\operatorname{Hist}_{i}\rightarrow V$. It assigns to each history $hv$, with $v\in V_{i}$, a vertex $v^{\prime}$ such that $(v,v^{\prime})\in E$. In an initialized game $(\mathcal{G},v_{0})$, $\sigma_{i}$ needs only to be defined for histories starting in $v_{0}$. We denote by $\Sigma_{i}$ the set of strategies for Player $i$. A play $\rho=\rho_{0}\rho_{1}\ldots$ is consistent with $\sigma_{i}$ if for all $\rho_{k}\in V_{i}$, $\sigma_{i}(\rho_{0}\ldots\rho_{k})=\rho_{k+1}$. A strategy $\sigma_{i}$ is positional if it only depends on the last vertex of the history, i.e., $\sigma_{i}(hv)=\sigma_{i}(v)$ for all $hv\in\operatorname{Hist}_{i}$. It is finite-memory if it can be encoded by a finite-state machine.

A strategy profile is a tuple $\sigma=(\sigma_{i})_{i\in\Pi}$ of strategies, one for each player. Given an initialized game $(\mathcal{G},v_{0})$ and a strategy profile $\sigma$, there exists an unique play from $v_{0}$ consistent with each strategy $\sigma_{i}$. We call this play the outcome of $\sigma$ and denote it by $\langle\sigma\rangle_{v_{0}}$. We say that $\sigma$ has cost profile $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}$.

2.0.2 Quantitative reachability games

In this article, we are interested in reachability games: each player has a target set of vertices that he wants to reach.

Definition 1

A quantitative reachability game $\mathcal{G}=(\mathcal{A},(\operatorname{Cost}_{i})_{i\in\Pi},(F_{i})_{i\in\Pi})$ is a game enhanced with a target set $F_{i}\subseteq V$ for each player $i\in\Pi$ and for all $i\in\Pi$ the cost function $\operatorname{Cost}_{i}$ is defined as follows: for all $\rho=\rho_{0}\rho_{1}\ldots\in\operatorname{Plays}$: $\operatorname{Cost}_{i}(\rho)=k$ if $k\in\mathbb{N}$ is the least index such that $\rho_{k}\in F_{i}$ and $\operatorname{Cost}_{i}(\rho)=+\infty$ if such index does not exist.

In quantitative reachability games, players have to pay a cost equal to the number of edges until visiting their own target set or $+\infty$ if it is not visited. Thus each player aims at minimizing his cost.

2.0.3 Solution concepts

In the multiplayer game setting, the solution concepts usually studied are equilibria. We recall the concepts of Nash equilibrium and subgame perfect equilibrium.

Let $\sigma=(\sigma_{i})_{i\in\Pi}$ be a strategy profile in an initialized game $(\mathcal{G},v_{0})$. When we highlight the role of player $i$, we denote $\sigma$ by $(\sigma_{i},\sigma_{-i})$ where $\sigma_{-i}$ is the profile $(\sigma_{j})_{j\in\Pi\setminus\{i\}}$. A strategy $\sigma^{\prime}_{i}\neq\sigma_{i}$ is a deviating strategy of Player $i$, and it is a profitable deviation for him if $\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}})>\operatorname{Cost}_{i}(\langle\sigma^{\prime}_{i},\sigma_{-i}\rangle_{v_{0}})$.

The notion of Nash equilibrium is classical: a strategy profile $\sigma$ in an initialized game $(\mathcal{G},v_{0})$ is a Nash equilibrium (NE) if no player has an incentive to deviate unilaterally from his strategy, i.e. no player has a profitable deviation.

Definition 2 (Nash equilibrium)

Let $(\mathcal{G},v_{0})$ be an initialized quantitative reachability game. The strategy profile $\sigma$ is an NE if for each $i\in\Pi$ and each deviating strategy $\sigma^{\prime}_{i}$ of Player $i$, we have $\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}})\leq\operatorname{Cost}_{i}(\langle\sigma^{\prime}_{i},\sigma_{-i}\rangle_{v_{0}})$.

When considering games played on graphs, a useful refinement of NE is the concept of subgame perfect equilibrium (SPE) which is a strategy profile that is an NE in each subgame. Formally, given a game ${\mathcal{G}}=(\mathcal{A},(\operatorname{Cost}_{i})_{i\in\Pi})$, an initial vertex $v_{0}$, and a history $hv\in\operatorname{Hist}(v_{0})$, the initialized game $(\mathcal{G}_{\restriction h},v)$ such that $\mathcal{G}_{\restriction h}=(\mathcal{A},(\operatorname{Cost}_{i\restriction h})_{i\in\Pi})$ where $\operatorname{Cost}_{i\restriction h}(\rho)=\operatorname{Cost}_{i}(h\rho)$ for all $i\in\Pi$ and $\rho\in V^{\omega}$ is called a subgame of $(\mathcal{G},v_{0})$. Notice that $(\mathcal{G},v_{0})$ is a subgame of itself. Moreover if $\sigma_{i}$ is a strategy for player $i$ in $(\mathcal{G},v_{0})$, then $\sigma_{i\restriction h}$ denotes the strategy in $(\mathcal{G}_{\restriction h},v)$ such that for all histories $h^{\prime}\in\operatorname{Hist}_{i}(v)$, $\sigma_{i\restriction h}(h^{\prime})=\sigma_{i}(hh^{\prime})$. Similarly, from a strategy profile $\sigma$ in $(\mathcal{G},v_{0})$, we derive the strategy profile $\sigma_{\restriction h}$ in $(\mathcal{G}_{\restriction h},v)$.

Definition 3 (Subgame perfect equilibrium)

Let $(\mathcal{G},v_{0})$ be an initialized game. A strategy profile $\sigma$ is an SPE in $(\mathcal{G},v_{0})$ if for all $hv\in\operatorname{Hist}(v_{0})$, $\sigma_{\restriction h}$ is an NE in $(\mathcal{G}_{\restriction h},v)$.

Clearly, any SPE is an NE and it is stated in Theorem 2.1 in [3] that there always exists an SPE (and thus an NE) in quantitative reachability games.

2.0.4 Studied problems

We conclude this section with the problems studied in this article. Let us first recall the concepts of social welfare and Pareto optimality. Let $(\mathcal{G},v_{0})$ be an initialized quantitative reachability game with $\mathcal{G}=(\mathcal{A},(\operatorname{Cost}_{i})_{i\in\Pi},(F_{i})_{i\in\Pi})$. Given $\rho=\rho_{0}\rho_{1}\ldots\in\operatorname{Plays}(v_{0})$, we denote by $\operatorname{Visit}(\rho)$ the set of players who visit their target set along $\rho$, i.e., $\operatorname{Visit}(\rho)=\{i\in\Pi\mid\text{there exists }n\in\mathbb{N}\text{ st. }\rho_{n}\in F_{i}\}$.111We can easily adapt this definition to histories. The social welfare of $\rho$, denoted by $\operatorname{SW}(\rho)$, is the pair $(|\operatorname{Visit}(\rho)|,\sum_{i\in\operatorname{Visit}(\rho)}\operatorname{Cost}_{i}(\rho))$. Note that it takes into account both the number of players who visit their target set and their accumulated cost to reach those sets. Finally, let $P=\{(\operatorname{Cost}_{i}(\rho))_{i\in\Pi}\mid\rho\in\operatorname{Plays}(v_{0})\}\subseteq(\mathbb{N}\cup\{+\infty\})^{|\Pi|}$. A cost profile $p\in P$ is Pareto optimal in $\operatorname{Plays}(v_{0})$ if it is minimal in $P$ with respect to the componentwise ordering $\leq$ on $P$ 222For convenience, we prefer to say that $p$ is Pareto optimal in $\operatorname{Plays}(v_{0})$ rather than in $P$..

Let us now state the studied decision problems. The first two problems are classical: they ask whether there exists a solution (NE or SPE) $\sigma$ satisfying certain requirements that impose bounds on either $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}$ or on $\operatorname{SW}(\langle\sigma\rangle_{v_{0}})$.

Problem 1 (Threshold decision problem)

Given an initialized quantitative reachability game $(\mathcal{G},v_{0})$, given a threshold $y\in(\mathbb{N}\cup\{+\infty\})^{|\Pi|}$, decide whether there exists a solution $\sigma$ such that $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}\leq y$.

The most natural requirements are to impose upper bounds on the costs that the players have to pay and no lower bounds. One might also be interested in imposing an interval $[x_{i},y_{i}]$ in which must lie the cost paid by Player $i$.

In the second problem, constraints are imposed on the social welfare, with the aim to maximize it. We use the lexicographic ordering on $\mathbb{N}^{2}$ such that $(k,c)\succeq(k^{\prime},c^{\prime})$ if and only if (i) $k\geq k^{\prime}$ or (ii) $k=k^{\prime}$ and $c\leq c^{\prime}$.

Problem 2 (Social welfare decision problem)

Given an initialized quantitative reachability game $(\mathcal{G},v_{0})$, given two thresholds $k\in\{0,\ldots,|\Pi|\}$ and $c\in\mathbb{N}$, decide whether there exists a solution $\sigma$ such that $\operatorname{SW}(\langle\sigma\rangle_{v_{0}})\succeq(k,c)$.

Notice that with the lexicographic ordering, we want to first maximize the number of players who visit their target set, and then to minimize the accumulated cost to reach those sets. Let us now state the last studied problem.

Problem 3 (Pareto optimal decision problem)

Given an initialized quantitative reachability game $(\mathcal{G},v_{0})$ decide whether there exists a solution $\sigma$ in $(\mathcal{G},v_{0})$ such that $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}$ is Pareto optimal in $\operatorname{Plays}(v_{0})$.

Remark 1

Problems 1 and 2 impose constraints with large inequalities. We could also impose strict inequalities or even a mix of strict and large inequalities. The results of this article can be easily adapted to those variants.

We conclude this section with an illustrative example.

Example 1

Consider the quantitative reachability game $(\mathcal{G},v_{0})$ of Figure 1. We have two players such that the vertices of Player $1$ (resp. Player $2$) are rounded (resp. rectangular) vertices. For the moment, the reader should not consider the value indicated on the right of the vertices’ labeling. Moreover $F_{1}=\{v_{3},v_{4}\}$ and $F_{2}=\{v_{1},v_{4}\}$. In this figure, an edge $(v,v^{\prime})$ labeled by $x$ should be understood as a path from $v$ to $v^{\prime}$ with length $x$. Observe that $F_{1}$ and $F_{2}$ are both reachable from the initial vertex $v_{0}$. Moreover the two Pareto optimal cost profiles are $(3,3)$ and $(2,6)$: take a play with prefix $v_{0}v_{2}v_{4}$ in the first case, and a play with prefix $v_{0}v_{2}v_{3}v_{0}v_{1}$ in the second case.

For this example, we claim that there is no NE (and thus no SPE) such that its cost profile is Pareto optimal (see Problem 3). Assume the contrary and suppose that there exists an NE $\sigma$ such that its outcome $\rho$ has cost profile $(3,3)$, meaning that $\rho$ begins with $v_{0}v_{2}v_{4}$. Then Player $1$ has a profitable deviation such that after history $v_{0}v_{2}$ he goes to $v_{3}$ instead of $v_{4}$ in a way to pay a cost of $2$ instead of $3$, which is a contradiction. Similarly assume that there exists an NE $\sigma$ such that its outcome $\rho$ has cost profile $(2,6)$, meaning that $\rho$ begins with $v_{0}v_{2}v_{3}v_{0}v_{1}$. Then Player $2$ has a profitable deviation such that after history $v_{0}$ he goes to $v_{1}$ instead of $v_{2}$, again a contradiction. So there is no NE $\sigma$ in $(\mathcal{G},v_{0})$ such that $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}$ is Pareto optimal in $\operatorname{Plays}(v_{0})$.

The previous discussion shows that there is no NE $\sigma$ such that $(0,0)=x\leq(\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}\leq y=(3,3)$ (see Problem 1). This is no longer true with $y=(6,3)$. Indeed, one can construct an NE $\tau$ whose outcome has prefix $v_{0}v_{1}v_{0}v_{2}v_{3}$ and cost profile $(6,3)$. This also shows that there exists an NE $\sigma$ (the same $\tau$ as before) that satisfies $\operatorname{SW}(\langle\sigma\rangle_{v_{0}})\succeq(k,c)=(2,9)$ (with $\tau$ both players visit their target set and their accumulated cost to reach it equals $9$). ∎

3 Existence problems

In this section, we show that for particular families of reachability games and requirements, there is no need to solve the related decision problems because they always have a positive answer in this case.

We begin with the family constituted by all reachability games with a strongly connected arena. The next theorem then states that there always exists a solution that visits all non empty target sets.

Theorem 3.1

Let $(\mathcal{G},v_{0})$ be an initialized quantitative reachability game such that its arena $\mathcal{A}$ is strongly connected. There exists an SPE $\sigma$ (and thus an NE) such that its outcome $\langle\sigma\rangle_{v_{0}}$ visits all target sets $F_{i}$, $i\in\Pi$, that are non empty.

Let us comment this result. For this family of games, the answer to Problem 1 is always positive for particular thresholds. In case of quantitative reachability, take strict constraints $<y_{i}=+\infty$ if $F_{i}\neq\emptyset$ and large constraints $\leq+\infty$ otherwise. The answer to Problem 2 is also always positive for threshold $k=|\{i\mid F_{i}\neq\emptyset\}|$ and $c=+\infty$.

In the statement of Theorem 3.1, as the arena is strongly connected, $F_{i}$ is non empty if and only if $F_{i}$ is reachable from $v_{0}$. Also notice that the hypothesis that the arena is strongly connected is necessary. Indeed, it is easy to build an example with two players (Player $1$ and Player $2$) such that from $v_{0}$ it is not possible to reach both $F_{1}$ and $F_{2}$.

We now turn to the second result of this section. The next theorem states that even with only two players there exists an initialized quantitative reachability game that has no NE with a cost profile which is Pareto optimal. To prove this result, we only have to come back to the quantitative reachability game of Figure 1. We explained in Example 1 that there is no NE in this game such that its cost profile is Pareto optimal.

Theorem 3.2

There exists an initialized quantitative reachability game with $|\Pi|=2$ that has no NE with a cost profile which is Pareto optimal in $\operatorname{Plays}(v_{0})$.

Notice that in the qualitative setting, in two-player games, there always exists an NE (resp. SPE) such that the gain profile333In the qualitative setting, each player obtain a gain that he wants to maximize: either 1 (if he visits his target set) or 0 (otherwise), all definitions are adapted accordingly. is Pareto optimal in $\operatorname{Plays}(v_{0})$ however this existence result cannot be extended to three players.

4 Solving decision problems

In this section, we provide the complexity results for the different problems without any assumption on the arena of the game. Even if we provide complexity lower bounds, the main part of our contribution is to give the upper bounds. Roughly speaking the decision algorithms work as follows: they guess a path and check that it is the outcome of an equilibrium satisfying the relevant property (such as Pareto optimality). In order to verify that a path is an equilibrium outcome, we rely on the outcome characterization of equilibria, presented in Section 4.2. These characterizations rely themselves on the notion of $\lambda$-consistent play, introduced in Section 4.1. As the guessed path should be finitely representable, we show that we can only consider $\lambda$-consistent lassoes, in Section 4.3. Finally, we expose the philosophy of the algorithms providing the upper bounds on the complexity of the three problems in Section 4.4.

4.1 $\lambda$-consistent play

We here define the labeling function, $\lambda:V\rightarrow\mathbb{N}\cup\{+\infty\}$ used to obtain the outcome characterization of equilibria. Given a vertex $v\in V$ along a play $\rho$, intuitively, the value $\lambda(v)$ represents the maximal number of steps within which the player who owns this vertex should reach his target set along $\rho$ starting from $v$. A play which satisfies the constraints given by $\lambda$ is called a $\lambda$-consistent play.

Definition 4 ($\lambda$-consistent play)

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game and $\lambda:V\rightarrow\mathbb{N}\cup\{+\infty\}$ be a labeling function. Let $\rho\in\operatorname{Plays}$ be a play, we say that $\rho=\rho_{0}\rho_{1}\ldots$ is $\lambda$-consistent if for all $i\in\Pi$ and all $k\in\mathbb{N}$ such that $i\not\in\operatorname{Visit}(\rho_{0}\ldots\rho_{k})$ and $\rho_{k}\in V_{i}$: $\operatorname{Cost}_{i}(\rho_{\geq k})\leq\lambda(\rho_{k}).$

The link between $\lambda$-consistency and equilibrium is made in Section 4.2.

Example 2

Let us come back to Example 1 and assume that the values indicated on the right of the vertices’ labeling represent the valuation of a labeling function $\lambda$. Let us first consider the play $\rho=(v_{0}v_{2}v_{4})^{\omega}$ with cost profile $(3,3)$. We have that $\operatorname{Cost}_{2}(\rho)=3\leq\lambda(v_{0})=3$ but $\operatorname{Cost}_{1}(\rho_{\geq 1})=\operatorname{Cost}_{1}(v_{2}v_{4}(v_{0}v_{2}v_{4})^{\omega})=2>\lambda(v_{2})=1$. This means that $(v_{0}v_{2}v_{4})^{\omega}$ is not $\lambda$-consistent. Secondly, one can easily see that the play $v_{0}v_{1}(v_{0}v_{2}v_{3})^{\omega}$ is $\lambda$-consistent.

4.2 Characterizations

4.2.1 Outcome characterization of Nash equilibria

To define the labeling function $\lambda$ which allows us to obtain this characterization, we need to study the rational behavior of one player playing against the coalition of the other players. In order to do so, with a quantitative reachability game $\mathcal{G}=(\mathcal{A},(\operatorname{Cost}_{i})_{i\in\Pi},(F_{i})_{i\in\Pi})$, we can associate $|\Pi|$ two-player zero-sum quantitative games [7]. For each $i\in\Pi$, we depict by $\mathcal{G}_{i}$ the (quantitative) coalitional game associated with Player $i$. In such a game Player $i$ (which becomes Player $Min$) wants to reach the target set $F=F_{i}$ within a minimum number of steps, and the coalition of all players except Player $i$ (which forms one player called Player $Max$, aka $-i$) aims to avoid it or, if it is not possible, maximize the number of steps until reaching $F$.

Given a coalitional game $\mathcal{G}_{i}$ and a vertex $v\in V$, the value of $\mathcal{G}_{i}$ from $v$, depicted by $\operatorname{Val}_{i}(v)$, allows us to know what is the lowest (resp. greatest) cost (resp. gain) that Player $Min$ (resp. Player $Max$) can ensure to obtain from $v$. Moreover, as quantitative coalitional games are determined these values always exist and can be computed in polynomial time [7, 8, 13].

An optimal strategy for Player $Min$ (resp. Player $Max$) in a coalitional game $\mathcal{G}_{i}$ is a strategy which ensures that, from all vertex $v\in V$, Player $Min$ (resp. Player $Max$) will pay (resp. obtain) at most $\operatorname{Val}_{i}(v)$ by following this strategy whatever the strategy of the other player. For each $i\in\Pi$, we know that there always exist optimal strategies for both players in $\mathcal{G}_{i}$. Moreover, we can always found optimal strategies which are positional [7].

In our characterization, we show that the outcomes of NEs are exactly the plays which are $\operatorname{Val}$-consistent, with the labeling function $\operatorname{Val}$ defined in this way: for all $v\in V$, $\operatorname{\operatorname{Val}}(v)=\operatorname{Val}_{i}(v)\quad\text{ if }v\in V_{i}.$

Theorem 4.1 (Characterization of NEs)

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game and let $\rho\in\operatorname{Plays}(v_{0})$ be a play, the next assertions are equivalent:

there exists an NE $\sigma$ such that $\langle\sigma\rangle_{v_{0}}=\rho$; 2. 2.

the play $\rho$ is $\operatorname{\operatorname{Val}}$-consistent.

Additionally, if $\rho=h\ell^{\omega}$ is a lasso, we can replace the first item by: there exists an NE $\sigma$ with memory in $\mathcal{O}(|h\ell|+|\Pi|)$ and such that $\langle\sigma\rangle_{v_{0}}=\rho$.

The main idea is that if the second assertion is false, then there exists a player $i$ who has an incentive to deviate along $\rho$. Indeed, if there exists $k\in\mathbb{N}$ such that $\operatorname{Cost}_{i}(\rho_{\geq k})>\operatorname{Val}_{i}(\rho_{k})$ ($\rho_{k}\in V_{i}$) it means that Player $i$ can ensure a better cost for him even if the other players play in coalition and in an antagonistic way. Thus, Player $i$ has a profitable deviation. For the second implication, the Nash equilibrium $\sigma$ is defined as follows: all players follow the outcome $\rho$ but if one player, assume it is Player $i$, deviates from $\rho$ the other players form a coalition $-i$ and punish the deviator by playing the optimal strategy of player $-i$ in the coalitional game $\mathcal{G}_{i}$. Thus, if $\rho=h\ell^{\omega}$, a player has to remember: (i) $h\ell$ to know both what he has to play and if someone has deviated and (ii) who is the deviator.

Example 3

Let us go back to Example 2, in this example the used labeling function $\lambda$ is in fact the labeling function $\operatorname{Val}$. We proved in Example 2 that the play $(v_{0}v_{2}v_{4})^{\omega}$ is not $\operatorname{Val}$-consistent and so not the outcome of an NE by Theorem 4.1. On the contrary, we have seen that the play $v_{0}v_{1}(v_{0}v_{2}v_{3})^{\omega}$ is $\operatorname{Val}$-consistent and it means that it is the outcome of an NE (again by Theorem 4.1). Notice that we have already proved these two facts in Example 1.

4.2.2 Outcome characterization of subgame perfect equilibria

In the previous section, we proved that the set of plays which are $\operatorname{\operatorname{Val}}$-consistent is equal to the set of outcomes of NEs. We now want to have the same kind of characterization for SPEs. We may not use the notion of $\operatorname{\operatorname{Val}}$-consistent plays because there exist plays which are $\operatorname{\operatorname{Val}}$-consistent but which are not the outcome of an SPE. But, we can recover the characterization of SPEs thanks to a different labeling function defined in [6] that we depict by $\lambda^{*}$. Notice that, $\lambda^{*}$ is not defined on the vertices of the game $\mathcal{G}$ but on the vertices of the extended game $\mathcal{X}$ associated with $\mathcal{G}$. Vertices in such a game are the vertices in $\mathcal{G}$ equipped with a subset of players who have already visited their target set. This game is also a reachability game thus all concepts and definitions introduced in Section 2 hold. Moreover, there is a one-to-one correspondence between SPEs in $\mathcal{G}$ and its extended game. This is the reason why we solve the different decision problems on the extended games $(\mathcal{X},x_{0})$, where $x_{0}=(v_{0},\operatorname{Visit}(v_{0}))$, instead of $(\mathcal{G},v_{0})$. More details are given in [6]. However, it is very important to notice that some of our results depend on $|V|$ (resp. $|\Pi|$) that are the number of vertices (resp. players) in $\mathcal{G}$ and not in $\mathcal{X}$.

Theorem 4.2 ([6] Characterization of SPEs)

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game and $(\mathcal{X},x_{0})$ be its extended game and let $\rho=\rho_{0}\rho_{1}\ldots\in\operatorname{Plays}(x_{0})$ be a play in the extended game, the next assertions are equivalent:

there exists a subgame perfect equilibrium $\sigma$ such that $\langle\sigma\rangle_{x_{0}}=\rho$; 2. 2.

the play $\rho$ is $\lambda^{*}$-consistent.

4.3 Sufficiency of lassoes

In this section, we provide technical results which given a $\lambda$-consistent play produce an associated $\lambda$-consistent lasso. In the sequel, we show that working with these lassoes is sufficient for the algorithms.

The associated lassoes are built by eliminating some unnecessary cycles and then identifying a prefix $h\ell$ such that $\ell$ can be repeated infinitely often. An unnecessary cycle is a cycle inside of which no new player visits his target set. More formally, let $\rho=\rho_{0}\rho_{1}\ldots\rho_{k}\ldots\rho_{k+\ell}\ldots$ be a play in $\mathcal{G}$, if $\rho_{k}=\rho_{k+\ell}$ and $\operatorname{Visit}(\rho_{0}\ldots\rho_{k})=\operatorname{Visit}(\rho_{0}\ldots\rho_{k+\ell})$ then the cycle $\rho_{k}\ldots\rho_{k+\ell}$ is called an unnecessary cycle.

We call: (P1) the procedure which eliminates an unnecessary cycle, i.e., let $\rho=\rho_{0}\rho_{1}\ldots\rho_{k}\ldots\rho_{k+\ell}\ldots$ such that $\rho_{k}\ldots\rho_{k+\ell}$ is an unnecessary cycle, $\rho$ becomes $\rho^{\prime}=\rho_{0}\ldots\rho_{k}\rho_{k+\ell+1}\ldots$ and (P2) the procedure which turns $\rho$ into a lasso $\rho^{\prime}=h\ell^{\omega}$ by copying $\rho$ long enough for all players to visit their target set and then to form a cycle after the last player has visited his target set. If no player visits his target set along $\rho$, then (P2) only copies $\rho$ long enough to form a cycle. Notice that, given $\rho\in\operatorname{Plays}$, applying (P1) or (P2) may involve a decreasing of the costs but for both $\operatorname{Visit}(\rho)=\operatorname{Visit}(\rho^{\prime})$ and for (P2) $\operatorname{Visit}(h)=\operatorname{Visit}(\rho^{\prime})$. Additionally, applying (P1) until it is no longer possible and then (P2), leads to a lasso with length at most $(|\Pi|+1)\cdot|V|$ and cost less than or equal to $|\Pi|\cdot|V|$ for players who have visited their target set.

Additionally, applying (P1) or (P2) on $\lambda$-consistent play preserves this property. It is stated in Lemma 1 which is in particular true for extended games.

Lemma 1

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game and $\rho\in\operatorname{Plays}$ be a $\lambda$-consistent play for a given labeling function $\lambda$. If $\rho^{\prime}$ is the play obtained by applying (P1) or (P2) on $\rho$, then $\rho^{\prime}$ is $\lambda$-consistent.

These properties on (P1) and (P2) allow us to claim that it is sufficient to deal with lassoes with polynomial length to solve Problems 1 and 3 for NEs and it give us some bounds on the needed memory and the costs for each problem.

Corollary 1

Let $\sigma$ be an NE (resp. SPE) in a quantitative reachability game $(\mathcal{G},v_{0})$ (resp. $(\mathcal{X},x_{0})$ its extended game) and $y\in(\mathbb{N}\cup\{+\infty\})^{|\Pi|}$. Let $w_{0}=v_{0}$ (resp. $w_{0}=x_{0}$). If $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{w_{0}}))_{i\in\Pi}\leq y$, then there exists $\tau$ an NE (resp. SPE) in $(\mathcal{G},v_{0})$ (resp. $(\mathcal{X},x_{0})$) such that:

• •

$(\operatorname{Cost}_{i}(\langle\tau\rangle_{w_{0}}))_{i\in\Pi}\leq y$;

• •

$\langle\tau\rangle_{w_{0}}$* is a lasso $h\ell^{\omega}$ such that $|h\ell|\leq(|\Pi|+1)\cdot|V|$;*

• •

for each $i\in\operatorname{Visit}(\langle\tau\rangle_{w_{0}})$, $\operatorname{Cost}_{i}(\langle\tau\rangle_{w_{0}})\leq|\Pi|\cdot|V|$;

• •

$\tau$* has memory in $\mathcal{O}((|\Pi|+1)\cdot|V|)$ (resp. $\mathcal{O}(2^{|\Pi|}\cdot|\Pi|\cdot|V|^{(|\Pi|+1)\cdot(|\Pi|+|V|)+1})$).*

Proposition 1

Let $(\mathcal{G},v_{0})$ (resp. $(\mathcal{X},x_{0})$ its extended game) be a quantitative reachability game and let $\sigma$ be an NE (resp. SPE). Let $w_{0}=v_{0}$ (resp. $w_{0}=x_{0}$). If we have that $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{w_{0}}))_{i\in\Pi}$ is Pareto optimal in $\operatorname{Plays}(w_{0})$, then:

• •

for all $i\in\operatorname{Visit}(\langle\sigma\rangle_{w_{0}})$, $\operatorname{Cost}_{i}(\langle\sigma\rangle_{w_{0}})\leq|V|\cdot|\Pi|;$

• •

there exists $\tau$ an NE (resp. SPE) such that $\langle\tau\rangle_{w_{0}}=h\ell^{\omega}$, $|h\ell|\leq(|\Pi|+1)\cdot|V|$ and $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{w_{0}}))_{i\in\Pi}=(\operatorname{Cost}_{i}(\langle\tau\rangle_{w_{0}}))_{i\in\Pi}$.

4.4 Algorithms

In this section, we provide the main ideas behind our algorithms.

To solve Problem 1444As Problem 1 is already solved in PSPACE for SPEs [6] we here focus only on NEs. (resp. Problem 3) for NEs, we use Corollary 1 (resp. Proposition 1) which ensures that if there exists an NE which satisfies the conditions555Satisfying the conditions is either satisfying the constraints (Problem 1 and Problem 2) or having a cost profile which is Pareto optimal (Problem 3)., there exists another one with a lasso outcome of polynomial length. The algorithm works as follows:(i) it guesses a lasso of polynomial length;(ii) it verifies that the cost profile of this lasso satisfies the conditions given by the problem (resp. is Pareto optimal in $\operatorname{Plays}(v_{0})$) and (iii) it verifies that the lasso is the outcome of an NE (Theorem 4.1). Notice that this latter step is done in polynomial time as the lasso has a polynomial length and the values of the coalitional games are computed in polynomial time.

To solve Problem 2 (resp. Problem 3 for SPEs), we use the algorithm designed for Problem 1. Each algorithm works as follows:(i) it guesses a cost profile $c$;(ii) it verifies that $c$ satisfies the conditions given by the problem and (iii) it checks, thanks to the algorithm for Problem 1, if there exists an equilibrium with cost profile smaller than $c$ (resp. equal to $c$).

Notice that for Problem 3, we need to have an oracle allowing us to know if $c$ is Pareto optimal. This leads us to study Problem 4 which lies in co-NP.

Problem 4

Given a reachability game $(\mathcal{G},v_{0})$ (resp. its extended game $(\mathcal{X},x_{0})$) and a lasso $\rho\in\operatorname{Plays}(v_{0})$ (resp. $\rho\in\operatorname{Plays}(x_{0})$), we want to decide if $(\operatorname{Cost}_{i}(\rho))_{i\in\Pi}$ is Pareto optimal in $\operatorname{Plays}(v_{0})$ (resp. $\operatorname{Plays}(x_{0})$).

4.5 Results

Thanks to the previous discussions in Section 4.4, we obtain the following results. Notice that we do not provide the proof for the NP-hardness (resp. PSPACE-hardness) as it is very similar to the one given in [10] (resp. [6]).

Theorem 4.3

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game.

• •

For NEs: Problem 1 and Problem 2 are NP-complete while Problem 3 is NP-hard and belongs to $\Sigma^{P}_{2}$.

• •

For SPEs: Problems 1, 2 and 3 are PSPACE-complete.

Theorem 4.4

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game.

• •

For NEs: for each decision problem, if its answer is positive, then there exists a strategy profile $\sigma$ with memory in $\mathcal{O}((|\Pi|+1)\cdot|V|)$ which satisfies the conditions.

• •

For SPEs: for each decision problem, if the answer is positive, then there exists a strategy profile $\sigma$ with memory in $\mathcal{O}(2^{|\Pi|}\cdot|\Pi|\cdot|V|^{(|\Pi|+1)\cdot(|\Pi|+|V|)+1})$ which satisfies the conditions.

• •

For both NEs and SPEs: (i) for Problem 1 and Problem 3, $\sigma$ is such that: if $i\in\operatorname{Visit}(\langle\sigma\rangle_{v_{0}})$, $\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}})\leq|\Pi|\cdot|V|$ and (ii) for Problem 2, $\sigma$ is such that: $\sum_{i\in\operatorname{Visit}(\langle\sigma\rangle_{v_{0}})}\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}})\leq|\Pi|^{2}\cdot|V|$.

Appendix 0.A Complements to Section 3

0.A.1 Proof of Theorem 3.1

To prove Theorem 3.1, we begin with a preliminary lemma and the proof of Theorem 3.1 follows.

Lemma 2

Let $\mathcal{G}$ be a quantitative reachability game. Then for all $v_{0}\in V$ for which some target set $F_{j}$, $j\in\Pi$, is reachable from $v_{0}$, there exists an SPE in $(\mathcal{G},v_{0})$ whose outcome $\rho$ visits at least one target set $F_{i}$, $i\in\Pi$, that is, $|\operatorname{Visit}(\rho)|\geq 1$.

Proof

By Theorem 2.1 in [3], there exists an SPE in $(\mathcal{G},v_{0})$ for each initial vertex $v_{0}\in V$. Consider the set $U\subseteq V$ of vertices $u$ for which some $F_{j}$ is reachable from $u$, and the set $U^{\prime}\subseteq U$ of those vertices $u$ for which there is an SPE in $(\mathcal{G},u)$ that visits at least one target set. We have to prove that $U=U^{\prime}$.

Assume the contrary and let $v_{0}\in U\setminus U^{\prime}$. We claim that there exists an edge $(u,u^{\prime})$ such that $u\in U\setminus U^{\prime}$ and $u^{\prime}\in U^{\prime}$. Indeed as $v_{0}\in U$, there exists a history $h=v_{0}v_{1}\ldots v_{k}$ with $v_{k}\in F_{j}$ for some $j$. Hence $v_{k}\in U^{\prime}$ since the outcome of all SPEs in $(\mathcal{G},v_{k})$ immediately visits $F_{j}$. As along $h$ we begin with $v_{0}\in U\setminus U^{\prime}$ and we end with $v_{k}\in U^{\prime}$, there must exist an edge $(v_{\ell},v_{\ell+1})=(u,u^{\prime})$ with $u\in U\setminus U^{\prime}$ and $u^{\prime}\in U^{\prime}$.

Let $\sigma^{u}$ (resp. $\sigma^{u^{\prime}}$) be an SPE in $(\mathcal{G},u)$ (resp. in $(\mathcal{G},u^{\prime})$). As $u^{\prime}\in U^{\prime}$, we can suppose that the outcome of $\sigma^{u^{\prime}}$ visits some target set $F_{j}$. From $\sigma^{u}$ and $\sigma^{u^{\prime}}$, we are going to construct another SPE $\tau$ in $(\mathcal{G},u)$ whose outcome will now visit this set $F_{j}$. This will lead to a contradiction with $u\in U\setminus U^{\prime}$. We define such a strategy profile $\tau$ equal to $\sigma^{u}$ except that it is replaced by $\sigma^{u^{\prime}}$ for all histories with prefix $uu^{\prime}$. More precisely,

• •

for the particular history $u$, if $u\in V_{i}$, then $\tau_{i}(u)=u^{\prime}$,

• •

for each history $uu^{\prime}h\in\operatorname{Hist}_{i}$, $i\in\Pi$, we define $\tau_{i}(uu^{\prime}h)=\sigma^{u^{\prime}}_{i}(u^{\prime}h)$,

• •

for each history $uv^{\prime}h\in\operatorname{Hist}_{i}$, $i\in\Pi$, with $v^{\prime}\neq u^{\prime}$, we define $\tau_{i}(uv^{\prime}h)=\sigma^{u}_{i}(uv^{\prime}h)$.

Clearly the outcome of $\tau$ is equal to $u\langle\sigma^{u^{\prime}}\rangle_{u^{\prime}}$ and thus visits $F_{j}$. It remains to show that $\tau$ is an SPE, i.e., that $\tau_{\restriction h}$ is an NE in the subgame $(\mathcal{G}_{\restriction h},v)$ for all $hv\in\operatorname{Hist}_{i}(v)$, $i\in\Pi$.

• •

For all histories $hv$ that begin with $uv^{\prime}$ with $v^{\prime}\neq u^{\prime}$, clearly $\tau_{\restriction h}$ is an NE in $(\mathcal{G}_{\restriction h},v)$ because $\tau_{\restriction h}=\sigma^{u}_{\restriction h}$ and $\sigma^{u}$ is an SPE.

• •

Take any history $hv$ that begin with $uu^{\prime}$, and let $h=uh^{\prime}$. Let $\tau^{\prime}_{i}$ be a deviating strategy for player $i$ in $(\mathcal{G}_{\restriction h},v)$. By definition of $\tau$ we have

[TABLE]

Moreover, as $u$ belongs to no target set, we have $\operatorname{Cost}_{i}(u\rho)=1+\operatorname{Cost}_{i}(\rho)$ for all plays $\rho\in\operatorname{Plays}(u^{\prime})$. It follows that if $\tau^{\prime}_{i}$ is a profitable deviation for player $i$ with respect to $\tau_{\restriction h}$, it is also a profitable deviation with respect to $\sigma^{u^{\prime}}_{\restriction h^{\prime}}$. The latter case never holds because $\sigma^{u^{\prime}}$ is an SPE (and in particular $\sigma^{u^{\prime}}_{\restriction h^{\prime}}$ is an NE). Therefore $\tau_{\restriction h}$ is an NE in $(\mathcal{G}_{\restriction h},v)$.

• •

It remains to consider the history $u$ and to prove that $\tau$ is an NE in $(\mathcal{G},u)$. From what has been gathered so far, only player $i$ such that $u\in V_{i}$ might have a profitable deviation by deviating at the initial vertex $u$ with a strategy $\tau^{\prime}_{i}$ such that $\tau^{\prime}_{i}(u)=v^{\prime}\neq u^{\prime}=\tau_{i}(u)$. Notice that since $u\in U\setminus U^{\prime}$, we have $\operatorname{Cost}_{i}(\langle\sigma^{u}\rangle_{u})=+\infty$ and since $\sigma^{u}$ is an SPE (and in particular an NE), we have $\operatorname{Cost}_{i}(\langle\tau^{\prime}_{i},\sigma^{u}_{-i}\rangle_{u})=+\infty$. Moreover as $\tau^{\prime}_{i}(u)=v^{\prime}\neq u^{\prime}$ and by definition of $\tau$, we have $\operatorname{Cost}_{i}(\langle\tau^{\prime}_{i},\sigma^{u}_{-i}\rangle_{u})=\operatorname{Cost}_{i}(\langle\tau^{\prime}_{i},\tau_{-i}\rangle_{u})=+\infty$. It follows that $\tau^{\prime}_{i}$ is not a profitable deviation for player $i$ with respect to $\tau$, and then $\tau$ is an NE in $(\mathcal{G},u)$.

∎

Proof (of Theorem 3.1)

Let $(\mathcal{G},v_{0})$, with $\mathcal{G}=(\mathcal{A},(\operatorname{Cost}_{i})_{i\in\Pi},(F_{i})_{i\in\Pi})$, be an initialized quantitative reachability game such that its arena is strongly connected. Assume by contradiction that there exists no SPE in $(\mathcal{G},v_{0})$ whose outcome visits all target sets $F_{i}$, $i\in\Pi$, that are non empty. By Theorem 2.1 in [3], there exists an SPE $\sigma$ in $(\mathcal{G},v_{0})$, and we take such an SPE $\sigma$ whose outcome $\rho=\langle\sigma\rangle_{v_{0}}$ visits a maximum number of target sets, say $F_{i_{1}},F_{i_{2}},\ldots,F_{i_{k}}$. Thus by assumption there exists at least one $F_{j}\neq\emptyset$ with $j\not\in\{i_{1},\ldots,i_{k}\}$ that is not visited by $\rho$. Thanks to Lemma 2, we are going to define from $\sigma$ another SPE $\tau$ in $(\mathcal{G},v_{0})$ whose outcome visits all $F_{i_{1}},\ldots,F_{i_{k}}$ as well as an additional target set. This will lead to a contradiction.

Consider a prefix $\rho_{0}\rho_{1}\ldots\rho_{\ell}$ of $\rho$ that visits all $F_{i_{1}},\ldots,F_{i_{k}}$. We denote it by $gu$ with $u=\rho_{\ell}$. From $\mathcal{G}$ we define the quantitative reachability game $\mathcal{G}^{\prime}=(\mathcal{A},(\operatorname{Cost}^{\prime}_{i})_{i\in\Pi},(F^{\prime}_{i})_{i\in\Pi})$ with the same arena $\mathcal{A}$ and such that $F^{\prime}_{i}=\emptyset$ if $i\in\{i_{1},\ldots,i_{k}\}$ and $F^{\prime}_{i}=F_{i}$ otherwise ($(\operatorname{Cost}^{\prime}_{i})_{i\in\Pi}$ is defined with respect to $(F^{\prime}_{i})_{i\in\Pi}$ as in Definition 1). Notice that $F^{\prime}_{j}=F_{j}$ is not empty and it is reachable from $u$ since $\mathcal{A}$ is strongly connected. Therefore by Lemma 2, there exists an SPE $\sigma^{\prime}$ in $(\mathcal{G}^{\prime},u)$ that visits at least one target set $F^{\prime}_{j^{\prime}}$. From $\sigma$ and $\sigma^{\prime}$, we define a strategy profile $\tau$ in $(\mathcal{G},v_{0})$ as follows: let $h\in\operatorname{Hist}_{i}(v_{0})$,

• •

if $h=guh^{\prime}$ for some $h^{\prime}$, then $\tau_{i}(h)=\sigma^{\prime}_{i}(uh^{\prime})$,

• •

otherwise $\tau_{i}(h)=\sigma_{i}(h)$.

Thus, $\tau$ acts as $\sigma$, except that after a history beginning with $gu$, it acts as $\sigma^{\prime}$. Clearly the outcome of $\tau$ is equal to $g\langle\sigma^{\prime}\rangle_{u}$ and thus visits $F^{\prime}_{j^{\prime}}=F_{j^{\prime}}$ in addition to $F_{i_{1}},\ldots,F_{i_{k}}$. It remains to show that $\tau$ is an SPE. Consider $hv\in\operatorname{Hist}_{i}(v_{0})$, $i\in\Pi$, and let us show that $\tau_{\restriction h}$ is an NE in $(\mathcal{G}_{\restriction h},v)$.

• •

If neither $hv$ is a prefix of $gu$ nor $gu$ is a prefix of $hv$, then $\tau_{\restriction h}=\sigma_{\restriction h}$ by definition of $\tau$, and $\tau_{\restriction h}$ is an NE in $(\mathcal{G}_{\restriction h},v)$ because $\sigma$ is an SPE in $(\mathcal{G},v_{0})$.

• •

If $gu$ is a prefix of $hv$, let $h^{\prime}$ such that $gh^{\prime}=h$. Suppose first that $hv$ visits $F_{i}$, then player $i$ has clearly no incentive to deviate in $(\mathcal{G}_{\restriction h},v)$. Suppose now that $hv$ does not visit $F_{i}$, then $i\not\in\{i_{1},\ldots,i_{k}\}$ and $F^{\prime}_{i}=F_{i}$ by definition of $\mathcal{G}^{\prime}$. Hence for all plays $\pi$ in $(\mathcal{G}_{\restriction h},v)$ that start in $v$, $h^{\prime}\pi$ is a play in $(\mathcal{G^{\prime}},u)$ that start in $u$, and we have $\operatorname{Cost}_{i}(h\pi)=|gu|+\operatorname{Cost}^{\prime}_{i}(h^{\prime}\pi)$. Hence by definition of $\tau$, a profitable deviation for player $i$ with respect to $\tau_{\restriction h}$ $(\mathcal{G}_{\restriction h},v)$ would be a profitable deviation with respect to $\sigma^{\prime}_{\restriction h^{\prime}}$ in $(\mathcal{G^{\prime}}_{\restriction h^{\prime}},v)$. The latter case cannot happen as $\sigma^{\prime}$ is an SPE in $(\mathcal{G}^{\prime},u)$ and it follows that $\tau_{\restriction h}$ is an NE in $(\mathcal{G}_{\restriction h},v)$.

• •

Consider the last case where $hv$ is a prefix of $gu$ with $hv\neq gu$, and let $hh^{\prime}=g$. Consider $\tau^{\prime}_{i}$ a deviating strategy for player $i$ with respect to $\tau_{\restriction h}$ in the subgame $(\mathcal{G}_{\restriction h},v)$, and let $\rho^{\prime}=\langle(\tau^{\prime}_{i},\tau_{\restriction h,-i})\rangle_{v}$. Without loss of generality, we can suppose that $h^{\prime}u$ is not a prefix of $\rho^{\prime}$ since this case was treated at the previous item. Notice that if $i\in\{i_{1},\ldots,i_{k}\}$, then $\operatorname{Cost}_{i}(\langle\tau_{\restriction h}\rangle_{v})=\operatorname{Cost}_{i}(\langle\sigma_{\restriction h}\rangle_{v})$, otherwise $\operatorname{Cost}_{i}(\langle\tau_{\restriction h}\rangle_{v})\leq+\infty=\operatorname{Cost}_{i}(\langle\sigma_{\restriction h}\rangle_{v})$. In both cases, as $h^{\prime}u$ is a prefix of both $\langle\tau_{\restriction h}\rangle_{v}$ and $\langle\sigma_{\restriction h}\rangle_{v}$, but not a prefix of $\rho^{\prime}$, if $\tau^{\prime}_{i}$ was a profitable deviation for player $i$ with respect to $\tau_{\restriction h}$, it would also be a profitable deviation with respect to $\sigma_{\restriction h}$ which is impossible since $\sigma$ is an SPE.

∎

We end with an example which shows that the hypothesis Theorem 3.1 that the arena is strongly connected is necessary.

Example 4

Consider the initialized qualitative reachability game $(\mathcal{G},v_{0})$ of Figure 2. There are two players, Player 1 who owns round vertices and Player 2 who owns square vertices, and $F_{1}=\{v_{1}\}$, $F_{2}=\{v_{2}\}$. Clearly there is a unique NE $\sigma=(\sigma_{1},\sigma_{2})$ in $(\mathcal{G},v_{0})$ such that $\sigma_{1}(v_{0})=v_{1}$ and $\sigma_{2}(v_{1})=v_{1}$, $\sigma_{2}(v_{2})=v_{2}$. Its outcome only visits $F_{1}$ (and not $F_{2}$).

∎

Appendix 0.B Complements to Section 4.2

0.B.1 Complements about coalitionnal games

We provide the formal definitions of (quantitative) coalitional game, value and optimal strategy.

Definition 5 ((Quantitative) Coalitional game)

Let $\mathcal{A}=(\Pi,V,E;(V_{i})_{i\in\Pi})$ be an arena and $\mathcal{G}=(\mathcal{A},(\operatorname{Cost}_{i})_{i\in\Pi},(F_{i})_{i\in\Pi})$ be a quantitative reachability game with $|\Pi|\geq 2$. With each player $i\in\Pi$, we can associate a two-player zero-sum quantitative reachability game depicted by $\mathcal{G}_{i}=(\mathcal{A}_{i},(\operatorname{\operatorname{Cost}_{Min}},\operatorname{\operatorname{Gain}_{Max}}),F)$ and defined as follows: i) $\mathcal{A}_{i}=(\{i,-i\},V,(V_{i},V\backslash V_{i}),E)$ where Player $i$ (resp. $-i$) can be called Player Min (resp. Player Max); ii) $\operatorname{\operatorname{Cost}_{Min}}=\operatorname{Cost}_{i}$ and $\operatorname{\operatorname{Gain}_{Max}}=\operatorname{\operatorname{Cost}_{Min}}$ and iii) $F=F_{i}$.

Definition 6 (Value)

Let $\mathcal{G}_{i}$ be a coalitional game and $v\in V$ be a vertex, we define the value of $\mathcal{G}_{i}$ from $v$ as :

[TABLE]

Remark that, as for each $i\in\Pi$ the coalitional game $\mathcal{G}_{i}$ is determined ([7]) and $\operatorname{\operatorname{Cost}_{Min}}=\operatorname{\operatorname{Gain}_{Max}}$, the equality (1) could be defined as $Val_{i}(v)=\sup_{\sigma_{2}\in\Sigma_{Max}}\inf_{\sigma_{1}\in\Sigma_{Min}}\operatorname{\operatorname{Gain}_{Max}}(\langle\sigma_{1},\sigma_{2}\rangle_{v})$.

Definition 7 (Optimal strategy)

Let $\mathcal{G}_{i}$ be a coalitional game, we say that $\sigma_{1}^{*}\in\Sigma_{Min}$ is an optimal strategy for player $Min$ if, for all $v\in V$, we have that: $\sup_{\sigma_{2}\in\Sigma_{Max}}\operatorname{\operatorname{Cost}_{Min}}(\langle\sigma_{1}^{*},\sigma_{2}\rangle_{v})\leq\operatorname{Val}_{i}(v)$. Similarly, we say that $\sigma_{2}^{*}\in\Sigma_{Max}$ is an optimal strategy for player $Max$ if, for all $v\in V$, we have that:

$\inf_{\sigma_{1}\in\Sigma_{Min}}\operatorname{\operatorname{Gain}_{Max}}(\langle\sigma_{1},\sigma_{2}^{*}\rangle_{v})\geq\operatorname{Val}_{i}(v)$.

0.B.2 Proof of Theorem 4.1

Proof (of Theorem 4.1)

Let us first recall that, for all $i\in\Pi$, the coalitional game $\mathcal{G}_{i}$ is determined and there are optimal positional strategies for both players $(\sigma^{*}_{i},\sigma^{*}_{-i})$. Moreover as $\operatorname{\operatorname{Cost}_{Min}}=\operatorname{\operatorname{Gain}_{Max}}$, for all $v\in V$, we have:

[TABLE]

From the optimal strategy $\sigma^{*}_{-i}$ in $\mathcal{G}_{i}$ we can extract a strategy $\sigma^{*}_{j,i}$ in $\mathcal{G}$. Notice also that even if $\sigma^{*}_{i}$ is a strategy in $\mathcal{G}_{i}$, we can use it as a strategy for Player $i$ in $\mathcal{G}$. Let us prove the equivalence between the two assertions.

$\mathbf{1\Rightarrow 2}$: Let $\sigma$ be a Nash equilibrium in $(\mathcal{G},v_{0})$ such that $\langle\sigma\rangle_{v_{0}}=\rho$. Let us assume by contradiction that there exist $i\in\Pi$ and $k\in\mathbb{N}$ such that $i\not\in\operatorname{Visit}(\rho_{0}\ldots\rho_{k})$ and $\rho_{k}\in V_{i}$ such that:

[TABLE]

Let $h=\rho_{0}\ldots\rho_{k-1}$, we can write:

[TABLE]

Additionally, by definition of value in a coalitional game and thanks to the fact that the optimal strategies are positional:

[TABLE]

where $\sigma^{*}_{i}$ is the optimal strategy of Player $i$ in $\mathcal{G}_{i}$ and $\sigma_{i}$ is an abuse of notation to depict the strategy of the coalition $-i=\Pi\backslash\{i\}$ which follows strategies $\sigma_{j}$ for all $j\neq i$.

By (2), (3) and (4), it follows that:

[TABLE]

As $i\not\in\operatorname{Visit}(h)$ by hypothesis, we can conclude that:

[TABLE]

This means that following $\sigma_{i}$ along $h$ and then $\sigma^{*}_{i}$ once he reaches $\rho_{k}$ is a profitable deviation for Player i. This concludes the proof.

$\mathbf{2\Rightarrow 1}$: Let $\tau$ be a strategy profile such that $\langle\tau\rangle_{v_{0}}=\rho$. From $\tau$ we aims to construct a Nash equilibrium with the same outcome. The main idea is the following one: first, all player play according to $\tau$. But if a player, let us call him Player $i$ deviates from $\tau_{i}$, the other players form a coalition and each of them plays their strategy obtained thanks to the strategy $\sigma^{*}_{-i}$ in $\mathcal{G}_{i}$.

In order to define properly the researched Nash equilibrium, we have to define a punishment function $P:\operatorname{Hist}(v_{0})\rightarrow\Pi\cup\{\perp\}$ which allows us to know who is the player who has deviated for the first time from $\tau$. So for all $h\in\operatorname{Hist}(v_{0})$, $P(h)=\perp$ if no player has yet deviated and $P(h)=i$ for some $i\in\Pi$ if Player $i$ is the first player who has deviated along $h$. We can define $P$ as follows: for the initial vertex $P(v_{0})=\perp$ and then for all history $hv\in Hist(v_{0})$ with $v\in V$:

[TABLE]

We now define $\sigma$. For all $i\in\Pi$ and for all $h\in\operatorname{Hist}_{i}(v_{0})$:

[TABLE]

It is clear that $\langle\sigma\rangle_{v_{0}}=\rho$. It remains to prove that $\sigma$ is a Nash equilibrium in $(\mathcal{G},v_{0})$. Let us assume that $\sigma$ is not an NE. It means that there exists a profitable deviation depicted by $\tilde{\sigma}_{i}$ for some player $i$. We chose $i$ such that $i$ is the first player who has profitable deviation from $\sigma$ along $\rho$. Let $\tilde{\rho}=\langle\tilde{\sigma}_{i},\sigma_{-i}\rangle_{v_{0}}$ the outcome such that Player $i$ plays his profitable deviation. As $\tilde{\sigma}_{i}$ is a profitable deviation we have:

[TABLE]

Moreover as $\rho$ and $\tilde{\rho}$ both begin in $v_{0}$, they have a common prefix. Let $hv\in\operatorname{Hist}_{i}$ this longest common prefix. We have that: $\rho=h\langle\sigma_{\restriction h}\rangle_{v}$ and $\tilde{\rho}=h\langle\tilde{\sigma_{i}}_{\restriction h},\sigma_{-i\restriction h}\rangle_{v}$. Notice that $i\not\in\operatorname{Visit}(hv)$. But, by definition of $\sigma$ and as the optimal strategies in $\mathcal{G}_{i}$ are positional, we can rewrite these two equalities as follows: $\rho=h\langle\tau_{\restriction h}\rangle_{v}$ and $\tilde{\rho}=h\langle\tilde{\sigma_{i}}_{\restriction h},(\sigma^{*}_{j,i})_{j\in\Pi\backslash\{i\}}\rangle_{v}.$ Additionally, thanks to the definition of the value in the coalitional game $\mathcal{G}_{i}$:

[TABLE]

By hypothesis, as $hv$ is a prefix of $\rho$ and $i\not\in\operatorname{Visit}(hv)$, we have that $\operatorname{Val}_{i}(v)\geq\operatorname{Cost}_{i}(\langle\tau_{\restriction h}\rangle_{v})$. Thus by (6), it follows that:

[TABLE]

And thanks to the definition of the cost function associated with quantitative reachability games, we have that:

[TABLE]

Thus, we can conclude that $\operatorname{Cost}_{i}(\tilde{\rho})\geq\operatorname{Cost}_{i}(\rho)$ which leads to a contradiction with (5). This concludes the proof. ∎

0.B.3 Finite machine which represents strategies in Theorem 4.1

A finite-state Machine ${\cal M}=(M,m_{0},\alpha_{u},\alpha_{n})$ is such that $M$ is a finite set of states (the memory of the strategy), $m_{0}\in M$ is the initial memory state, $\alpha_{u}\colon M\times V\rightarrow M$ is the update function, and $\alpha_{n}\colon M\times V_{i}\rightarrow V$ is the next-action function. The machine $\cal M$ defines a strategy $\sigma_{i}$ such that $\sigma_{i}(hv)=\alpha_{n}(\widehat{\alpha}_{u}(m_{0},h),v)$ for all histories $hv\in\operatorname{Hist}_{i}$, where $\widehat{\alpha}_{u}(m,\epsilon)=m$ and $\widehat{\alpha}_{u}(m,hv)=\alpha_{u}(\widehat{\alpha}_{u}(m,h),v)$ for all $m\in M$ and $hv\in\operatorname{Hist}$. The size of the strategy $\sigma_{i}$ is the size $|M|$ of its machine $\cal M$. Note that $\sigma_{i}$ is positional when $|M|=1$.

Formally, let $\rho=\rho_{0}\ldots\rho_{k-1}(\rho_{k}\ldots\rho_{n})^{\omega}$, we define for all $i\in\Pi$ a finite-state machine $\mathcal{M}_{i}=(M,m_{0},\alpha_{u},\alpha_{n})$ where:

• •

$M=\{\rho_{0}\rho_{0},\rho_{0}\rho_{1},\ldots,\rho_{n-1}\rho_{n},\rho_{n}\rho_{k}\}\cup\Pi$.

The set $\{\rho_{0}\rho_{0},\rho_{0}\rho_{1},\ldots,\rho_{n-1}\rho_{n},\rho_{n}\rho_{k}\}$ allows us to be sure that the outcome $\rho$ is well followed by all the players. Once it is no longer the case, we only have to retain who has deviated, this is the role of $\Pi$. Notice that, even if we can have $\rho_{m}\rho_{m+1}=\rho_{m^{\prime}}\rho_{m^{\prime}+1}$ along $\rho$, the edges $\rho_{m}\rho_{m+1}$ and $\rho_{m^{\prime}}\rho_{m^{\prime}+1}$ are depicted by two different memory state in $M$. Thus $|M|=|h\ell|+2+|\Pi|$.

• •

$m_{0}=\rho_{0}\rho_{0}$ is the memory state which specifies that the plays has not begun yet.

• •

$\alpha_{u}:M\times V\rightarrow M$ is defined as follows: for all $m\in M$ and $v\in V$:

[TABLE]

• •

$\alpha_{n}:M\times V_{i}\rightarrow V$ is defined in this way: for all $m\in M$ and $v\in V_{i}$:

[TABLE]

0.B.4 Complements about extended game

We here provide the formal definition of an extended game.

Definition 8 (Extended game)

Let $\mathcal{G}=(\mathcal{A},(\operatorname{Cost}_{i})_{i\in\Pi},(F_{i})_{i\in\Pi})$ be a quantitative reachability game with an arena $\mathcal{A}=(\Pi,V,E,(V_{i})_{i\in\Pi})$, and let $v_{0}$ be an initial vertex. The extended game of $\mathcal{G}$ is equal to $\operatorname{\mathcal{X}}=(X,(\operatorname{Cost}^{X}_{i})_{i\in\Pi},(F^{X}_{i})_{i\in\Pi})$ with the arena $X=(\Pi,V^{X},E^{X},(V^{X}_{i})_{i\in\Pi})$, such that:

• •

$V^{X}=V\times 2^{\Pi}$

• •

$((v,I),(v^{\prime},I^{\prime}))\in E^{X}$ if and only if $(v,v^{\prime})\in E$ and $I^{\prime}=I\cup\{i\in\Pi\mid v^{\prime}\in F_{i}\}$

• •

$(v,I)\in V^{X}_{i}$ if and only if $v\in V_{i}$

• •

$(v,I)\in F^{X}_{i}$ if and only if $i\in I$

• •

for each $\rho\in\operatorname{Plays}_{X}$, $\operatorname{Cost}^{X}_{i}(\rho)$ is equal to the least index $k$ such that $\rho_{k}\in F^{X}_{i}$, and to $+\infty$ if no such index exists.

The initialized extended game $(\operatorname{\mathcal{X}},x_{0})$ associated with the initialized game $({\mathcal{G}},v_{0})$ is such that $x_{0}=(v_{0},I_{0})$ with $I_{0}=\{i\in\Pi\mid v_{0}\in F_{i}\}$.

Appendix 0.C Complements to Section 4.3

Lemma 3

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game and $\rho\in\operatorname{Plays}$ be a play.

• •

If $\rho^{\prime}$ is obtained by applying (P1) on $\rho$, then $(\operatorname{Cost}_{i}(\rho^{\prime}))_{i\in\Pi}\leq(\operatorname{Cost}_{i}(\rho))_{i\in\Pi}$

• •

If $\rho^{\prime}$ is obtained by applying (P2) on $\rho$, then $(\operatorname{Cost}_{i}(\rho^{\prime}))_{i\in\Pi}=(\operatorname{Cost}_{i}(\rho))_{i\in\Pi}$.

• •

Applying (P1) until it is no longer possible and then (P2), leads to a lasso $\rho^{\prime}$ with length at most $(|\Pi|+1)\cdot|V|$ and $\operatorname{Cost}_{i}(\rho^{\prime})\leq|V|\cdot|\Pi|$ for each $i\in\operatorname{Visit}(\rho^{\prime})$.

Remark 2 (about Lemma 3)

Notice that, given a quantitative reachability game $(\mathcal{G},v_{0})$, as its extended game $(\mathcal{X},x_{0})$ is in particular a quantitative reachability game, all statements of Lemma 3 hold.

But, if we only apply the third assertion on $(\mathcal{X},x_{0})$, we obtain bounds on the size of the lasso and on the cost of plays which depends on $|V^{X}|$. It means that it is exponential on the size of the initial game $\mathcal{G}$.

In fact, even for the extended game $(\mathcal{X},x_{0})$ we can obtain the that: applying (P1) until it is no longer possible and then (P2), leads to a lasso $\rho^{\prime}$ with size at most $(|\Pi|+1)\cdot|V|$ and $\operatorname{Cost}_{i}(\rho^{\prime})\leq|V|\cdot|\Pi|$ for each $i\in\operatorname{Visit}(\rho^{\prime})$ where $|V|$ is the number of vertices in $\mathcal{G}$.

0.C.1 Proof of Lemma 1

Proof (of Lemma 1)

We begin by proving the assertion for (P1). Let $\rho$ be a $\lambda$-consistent play and we apply (P1) on it to obtain $\rho^{\prime}$. Then, for all $i\in\Pi$, $\operatorname{Cost}_{i}(\rho^{\prime})\leq\operatorname{Cost}_{i}(\rho)$. And in particular, for all $i\in\Pi$ and for all $n\in\mathbb{N}$,

[TABLE]

where $\varphi$ is the injective function which matches a node in $\rho^{\prime}$ with its corresponding node in $\rho$, i.e., for all $n\in\mathbb{N}$, $\rho^{\prime}_{n}=\rho_{\varphi(n)}$. Let $i\in\Pi$ and $k\in\mathbb{N}$ such that $i\not\in\operatorname{Visit}(\rho^{\prime}_{0}\ldots\rho^{\prime}_{k})$ and $\rho^{\prime}_{k}\in V_{i}$, we have to prove that $\operatorname{Cost}_{i}(\rho^{\prime}_{\geq k})\leq\lambda(\rho^{\prime}_{k})$. By construction of $\rho^{\prime}$ and by (7), we have that

[TABLE]

But, as $\rho$ is $\lambda$-consistent, we also have that:

[TABLE]

and we can conclude that $\operatorname{Cost}_{i}(\rho^{\prime}_{\geq k})\leq\lambda(\rho^{\prime}_{k})$ which proves that $\rho^{\prime}$ is $\lambda$-consistent. This concludes the proof.

For (P2), the assertion holds because $\rho^{\prime}$ is a copy of $\rho$ until each player in $\operatorname{Visit}(\rho)$ has visited his target set and $\operatorname{Visit}(\rho)=\operatorname{Visit}(\rho^{\prime})$. ∎

0.C.2 Proof of Corollary 1

0.C.2.1 Proof of Corollary 1 for NEs

The proof of Corollary 1 for NEs is due to Corollary 2 that we prove below.

Corollary 2

Let $\sigma$ be an NE in a quantitative reachability game $(\mathcal{G},v_{0})$, then there exists $\tau$ an NE in $(\mathcal{G},v_{0})$ such that:

• •

$\langle\tau\rangle_{v_{0}}$* is a lasso $h\ell^{\omega}$ such that $|h\ell|\leq(|\Pi|+1)\cdot|V|$;*

• •

for all $i\in\operatorname{Visit}(\langle\tau\rangle_{v_{0}})$, $\operatorname{Cost}_{i}(\langle\tau\rangle_{v_{0}})\leq\min\{\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}}),|\Pi|\cdot|V|\}$ and for all $i\not\in\operatorname{Visit}(\langle\tau\rangle_{v_{0}})$, $\operatorname{Cost}_{i}(\langle\tau\rangle_{v_{0}})=\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}})=+\infty$;

• •

the memory of $\tau$ is in $\mathcal{O}((|\Pi|+1)\cdot|V|)$.

Proof (of Corollary 2)

Let $\rho$ be a play in $(\mathcal{G},v_{0})$ such that $\rho=\langle\sigma\rangle_{v_{0}}$. We apply procedure (P1) on $\rho$ until there is no longer an unnecessary cycle and then we apply (P2). In this way, we obtain a lasso $\rho^{\prime}=h\ell^{\omega}\in\operatorname{Plays}(v_{0})$. By Lemma 3, $|h\ell|\leq(|\Pi|+1)\cdot|V|$ and $\operatorname{Cost}_{i}(h\ell^{\omega})\leq\min\{\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}}),|\Pi|\cdot|V|\}$ if $i\in\operatorname{Visit}(\langle\sigma\rangle_{v_{0}})$ and $\operatorname{Cost}_{i}(h\ell^{\omega})=+\infty$ otherwise.

By hypothesis and thanks to Theorem 4.1, we know that $\rho$ is $\operatorname{\operatorname{Val}}$-consistent. Thus, by Lemma 1, $\rho^{\prime}$ is $\operatorname{\operatorname{Val}}$-consistent. And Theorem 4.1 for $\rho^{\prime}$ concludes the proof. ∎

Proof (of Corollary 1 for NEs)

It is a direct consequence of Corollary 2. ∎

0.C.2.2 Proof of Corollary 1 for SPEs

In this section we assume that all the definitions and notations introduced in [6] are known.

By adapting the concept of (good) symbolic witness (a set of lassoes with some good properties) used in [4], we can show that if there exists an SPE with a cost profile $c$ then, there exists one with the same cost profile but with a finite-memory. This leads to Proposition 2 which allows us to prove Corollary 1 for SPEs.

Before the statement of the proposition, we formally introduce what is a (good) symbolic witness. This notion was introduced in [4] for games with prefix-independent gain functions. We adapt it for quantitative reachability games for which the cost function is not prefix-independent.

Definition 9 (Symbolic witness)

Let $(\mathcal{G},v_{0})$ be an initialized quantitative reachability game and $(\mathcal{X},(v_{0},I_{0}))$ its extended game. Let $\mathcal{I}$ be a subset of $(\Pi\cup\{0\})\times V\times 2^{\Pi}$ such that:

[TABLE]

A symbolic witness is a set $\mathcal{P}=\{\rho_{i,v,I}\mid(i,v,I)\in\mathcal{I}\}$ such that each $\rho_{i,v,I}$ is a lasso in $\mathcal{X}$ with $\operatorname{First}(\rho_{i,v,I})=(v,I)$.

Definition 10 (Good symbolic witness)

A symbolic witness $\mathcal{P}$ is good if for all $\rho_{j,u,J}$, $\rho_{i,v^{\prime},I^{\prime}}\in\mathcal{P}$, for all suffix $\rho\in\operatorname{Plays}(v,I)$ of $\rho_{j,u,J}$ such that $((v,I),(v^{\prime},I^{\prime}))\in E^{X}$ and $(v,I)\in V_{i}^{X}$, if $i\not\in I$, then we have:

[TABLE]

Proposition 2

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game and $(\mathcal{X},(v_{0},I_{0}))$ be its extended game, let $c\in(\mathbb{N}\cup\{+\infty\})^{|\Pi|}$ and let $M=\max_{i\in\Pi}\{c_{i}\mid c_{i}<+\infty\}$ if this max exists, $M=0$ otherwise. The following assertions are equivalent:

There exists an SPE with cost profile $c$ in $(\mathcal{X},(v_{0},I_{0}))$; 2. 2.

$\Lambda^{*}(v,I)=\{\rho\in\operatorname{Plays}_{X}(v,I)\mid\rho\text{ is }\lambda^{*}\text{-consistent}\}$* $\neq\emptyset$ for all $(v,I)\in\operatorname{Succ}^{*}(v_{0},I_{0})$ and there exists $\rho\in\Lambda^{*}(v_{0},I_{0})$ such that $(\operatorname{Cost}_{i}(\rho))_{i\in\Pi}=c$;* 3. 3.

There exists a good symbolic witness $\mathcal{P}$ that contains a lasso $\rho_{0,v_{0},I_{0}}$ with cost profile $c$ and $|\rho_{0,v_{0},I_{0}}|\leq M+|V|$. Moreover, for each $\rho_{i,v,I}\in\mathcal{P}$, $|\rho_{i,v,I}|\leq\mathcal{O}(|V|^{(|\Pi|+1)\cdot(|\Pi|+|V|)})+(|\Pi|+1)\cdot|V|$; 4. 4.

There exists a finite-memory SPE $\sigma$ with cost profile $c$ in $(\mathcal{X},(v_{0},I_{0}))$ such that its memory is in $\mathcal{O}(M+2^{|\Pi|}\cdot|\Pi|\cdot|V|^{(|\Pi|+1)\cdot(|\Pi|+|V|)+1})$.

To obtain the bound on the length of the lassoes, we use the following lemma.

Lemma 4 ([6])

Let $v$ be a vertex in the extended game, let $\operatorname{MaxCost}_{i}(v)=\max\{\operatorname{Cost}_{i}(\rho)\mid\rho\in\operatorname{Plays}(v)\text{ and }\rho\text{ is }\lambda^{*}\text{-consistent}\}$. If $\operatorname{MaxCost}_{i}(v)<+\infty$, then $\operatorname{MaxCost}_{i}(v)\leq\mathcal{O}(|V|^{(|\Pi|+1)\cdot(|\Pi|+|V|)})$.

Notice that the proofs of $(2\Rightarrow 3)$ and $(3\Rightarrow 4)$ are quietly the same that the one of Proposition 33 in [6]. But, in the following proof, we choose more adequatly the plays of the form $\rho_{i,v^{\prime},I^{\prime}}$ by picking lassoes with a finite size in the sets of $\lambda^{*}$-consistent plays beginning in $(v^{\prime},I^{\prime})$. It allows us to build and highlight the existent of a symbolic witness which is a symbolic finite representation of an SPE who has cost profile equal to $c$. In this way we build an SPE with a finite memory.

Proof (of Proposition 2)

$\underline{1\Rightarrow 2}$: Theorem 17 in [6].

$\underline{2\Rightarrow 3}$: We build a symbolic witness $\mathcal{P}$ step by step and then prove that it is good. At the initialization, $\mathcal{P}=\emptyset$.

Let $\rho\in\Lambda^{*}(v_{0},I_{0})$ such that $(\operatorname{Cost}_{i}(\rho))_{i\in\Pi}=c$. We apply (P2) on $\rho$ to obtain a lasso $\rho_{0,v_{0},I_{0}}$ such that $|\rho_{0,v_{0},I_{0}}|\leq M+|V|$ and $(\operatorname{Cost}_{i}(\rho_{0,v_{0},I_{0}}))_{i\in\Pi}=c$. Moreover, as $\rho$ is $\lambda^{*}$-consistent, $\rho_{0,v_{0},I_{0}}$ is also $\lambda^{*}$-consistent (by Lemma 1). We add $\rho_{0,v_{0},I_{0}}$ to $\mathcal{P}$.

For each $(i,v,I)\in\mathcal{I}$, let $\displaystyle\rho=\operatorname*{argmax}_{\rho^{\prime}\in\Lambda^{*}(v,I)}\{\operatorname{Cost}_{i}(\rho^{\prime})\}$. We obtain $\rho_{i,v,I}$ by copying $\rho$ until Player $i$ has visited his target set, then by removing the unnecessary cycles and apply (P2). If Player $i$ does not visit his target set along $\rho$, we remove all the unnecessary cycles (by apply iteratively (P1)) and then we apply (P2). By the same kind of arguments than for Lemma 3 and Lemma 1, we obtain that: i) $\rho_{i,v,I}$ is $\lambda^{*}$-consistent, ii) $\operatorname{Cost}_{i}(\rho_{i,v,I})=\operatorname{Cost}_{i}(\rho)$ and iii) $|\rho_{i,v,I}|\leq\mathcal{O}(|V|^{(|\Pi|+1)\cdot(|\Pi|+|V|)})+(|\Pi|+1)\cdot|V|$ (by Lemma 4). We add $\rho_{i,v,I}$ to $\mathcal{P}$.

By construction $\mathcal{P}$ is a symbolic witness. It remains to prove that it is good.

Let $\rho_{j,u,J}$ and $\rho_{i,v^{\prime},I^{\prime}}\in\mathcal{P}$ and let $\rho\in\operatorname{Plays}(v,I)$ be a suffix of $\rho_{j,u,J}$ such that $((v,I),(v^{\prime},I^{\prime}))\in E^{X}$ and $(v,I)\in V^{X}_{i}$, we have to prove that if $i\not\in I$, $\operatorname{Cost}_{i}(\rho)\leq 1+\operatorname{Cost}_{i}(\rho_{i,v^{\prime},I^{\prime}})$.

As $i\not\in I$, $\displaystyle\lambda^{*+1}(v,I)=1+\min_{((v,I),(v^{\prime},I^{\prime})\in E^{X})}\{\operatorname{Cost}_{i}(\rho^{\prime})\mid\rho^{\prime}\in\Lambda^{*}(v^{\prime},I^{\prime})\}$, we have that $\lambda^{*}(v,I)=\lambda^{*+1}(v,I)\leq 1+\operatorname{Cost}_{i}(\rho_{i,v^{\prime},I^{\prime}})$. Finally, as $\rho_{j,u,J}$ is $\lambda^{*}$-consistent, we have:

[TABLE]

It concludes the proof.

$\underline{3\Rightarrow 4}$: Given a good symbolic witness $\mathcal{P}$ with properties given in statement 3, we show how to build an SPE with a finite memory.

We define a strategy profile $\sigma$ step by step by induction on the subgames of $(\mathcal{X},(v_{0},I_{0}))$. We first partially define $\sigma$ such that $\langle\sigma\rangle_{(v_{0},I_{0})}=\rho_{0,v_{0},I_{0}}$.

Consider next, $h(v,I)(v^{\prime},I^{\prime})\in\operatorname{Hist}(v_{0},I_{0})$ with $(v,I)\in V_{i}^{X}$ such that $\langle\sigma_{\restriction h}\rangle_{(v,I)}$ is already built but not $\langle\sigma_{\restriction h(v,I)}\rangle_{(v^{\prime},I^{\prime})}$. Then we extend $\sigma$ such that $\langle\sigma_{\restriction h(v,I)}\rangle_{(v^{\prime},I^{\prime})}=\rho_{i,v^{\prime},I^{\prime}}$.

Let us prove that $\sigma$ is a very weak SPE (and so an SPE). Consider the subgame $(\mathcal{X}_{\restriction h},(v,I))$ (with $(v,I)\in V_{i}^{X}$) and the one-shot deviating strategy $\sigma^{\prime}_{i}$ from $\sigma_{i\restriction h}$ such that $\sigma^{\prime}_{i}(v,I)=(v^{\prime},I^{\prime})$. By construction, there exists $\rho_{j,u,J}$ and $\rho_{i,v^{\prime},I^{\prime}}\in\mathcal{P}$ and $h^{\prime}\in\operatorname{Hist}(v_{0},I_{0})$ such that $h\langle\sigma_{\restriction h}\rangle_{(v,I)}=h^{\prime}\rho_{j,u,J}$ and $\langle\sigma_{\restriction h(v,I)}\rangle_{(v^{\prime},I^{\prime})}=\rho_{i,v^{\prime},I^{\prime}}.$ We have to prove that:

[TABLE]

If $i\in I$, then $\operatorname{Cost}_{i}(h\langle\sigma_{\restriction h}\rangle_{(v,I)})=\operatorname{Cost}_{i}(h(v,I)\langle\sigma_{\restriction h(v,I)}\rangle_{(v^{\prime},I^{\prime})})=0$. Otherwise,

[TABLE]

Notice that $\sigma$ has a cost profile equal to $c$ by construction, as $(\operatorname{Cost}_{i}(\rho_{0,v_{0},I_{0}}))_{i\in\Pi}=c$. It remains to prove that $\sigma$ is finite-memory with memory in $\mathcal{O}(M+2^{|\Pi|}\cdot|\Pi|\cdot|V|^{(|\Pi|+1)\cdot(|\Pi|+|V|)+1})$. Having $(j,u,J)$ in memory (the last deviating player $j$ and the vertex $(u,J)$ where he moved), the machine $\mathcal{M}_{i}$, $i\in\Pi$, which represents the strategy $\sigma_{i}$, has to produce the lasso $\rho_{j,u,J}$ of length bounded by $M+(|\Pi|+1)\cdot|V|$ for $\rho_{0,v_{0},I_{0}}$ and by $\mathcal{O}(|V|^{(|\Pi|+1)\cdot(|\Pi|+|V|)})$ for the others (at most $|\Pi|\cdot|V|\cdot 2^{|\Pi|}$ such lassoes). It leads to a memory in $\mathcal{O}(M+2^{|\Pi|}\cdot|\Pi|\cdot|V|^{(|\Pi|+1)\cdot(|\Pi|+|V|)+1})$.

$\underline{4\Rightarrow 1}$: Obvious. ∎

Proof (of Corollary 1 for SPEs)

By hypothesis, we know that there exists an SPE $\sigma$ such that $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{x_{0}}))\leq y$. Let $\rho=\langle\sigma\rangle_{x_{0}}$, and apply (P1) as long as possible and then (P2) in order to obtain a lasso $\rho^{\prime}=h\ell^{\omega}$ with size at most $(|\Pi|+1)\cdot|V|$ and such that $\operatorname{Cost}_{i}(\rho^{\prime})\leq\min\{|V|\cdot|\Pi|,y_{i}\}$ (Lemma 3). As $\rho$ is the outcome of an SPE, $\rho$ is $\lambda^{*}$-consistent and thus $\rho^{\prime}$ is also $\lambda^{*}$-consistent (Lemma 1). Thus $\rho^{\prime}$ is the outcome of an SPE.

Thanks to Proposition 2, we know that there exists an SPE $\tau$ with finite-memory in $\mathcal{O}(2^{|\Pi|}\cdot|\Pi|\cdot|V|^{(|\Pi|+1)\cdot(|\Pi|+|V|)+1})$ which as the same cost profile as $\rho^{\prime}$. So, $\operatorname{Cost}_{i}(\langle\tau\rangle_{x_{0}})\leq|V|\cdot|\Pi|$ for each $i\in\operatorname{Visit}(\langle\tau\rangle_{x_{0}})$ and $\tau$ fulfills the constraints. Moreover, one can assume that $\langle\tau\rangle_{x_{0}}$ is a lasso of length at most $(|\Pi|+1)\cdot|V|$. ∎

0.C.3 Proof of Proposition 1

Proof (of Proposition 1)

Second item is a direct consequence of the first one. Thus, let us prove first item.

Let $\sigma$ be an NE such that its cost profile is Pareto optimal in $\operatorname{Plays}(v_{0})$. To get a contradiction, assume that there exists $i\in\operatorname{Visit}(\langle\sigma\rangle_{v_{0}})$ such that $\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}})>|V|\cdot|\Pi|$. It means that there exists an unnecessary cycle before Player $i$ reaches his target set. By removing this cycle (applying (P1)), we obtain a new play $\rho^{\prime}$ such that $\operatorname{Cost}_{i}(\rho^{\prime})<\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}})$ and for Player $j$ ($j\neq i$), $\operatorname{Cost}_{j}(\rho^{\prime})\leq\operatorname{Cost}_{j}(\langle\sigma\rangle_{v_{0}})$ (by Lemma 3). It leads to a contradiction with the fact that $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}$ is Pareto optimal in $\operatorname{Plays}(v_{0})$.

The same proof holds for SPE. ∎

Appendix 0.D Complements of Section 4.5

0.D.1 Proof of Theorem 4.3 and Theorem 4.4 for Problem 1

Proposition 3

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game, for NE:

• •

Problem 1 is NP-easy.

• •

If the answer to this decision problem is positive, then there exists an NE $\sigma$ with memory in $\mathcal{O}((|\Pi|+1)\cdot|V|)$ which satisfies the constraints and such that for all $i\in\operatorname{Visit}(\langle\sigma\rangle_{v_{0}})$, $\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}})\leq|V|\cdot|\Pi|$.

Proof

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game and let $y\in(\mathbb{N}\cup\{+\infty\})^{|\Pi|}$ be a threshold. By Corollary 1, we know that if there exists an NE $\sigma$ which fulfills the constraints, then there exists an other one such that its outcome is of the form $h\ell^{\omega}$ with $|h\ell|\leq(|\Pi|+1)\cdot|V|$ and which fulfills the constraints too.

Thus, the NP-algorithm is the following one:

Step 1: Guess a lasso $\rho=h\ell^{\omega}$ with size at most $(|\Pi|+1)\cdot|V|$ such that $\operatorname{Visit}(\rho)=\operatorname{Visit}(h)$ and verify that $\rho$ is a lasso.

Step 2: Compute $(\operatorname{Cost}_{i}(\rho))_{i\in\Pi}$ and verify that $(\operatorname{Cost}_{i}(\rho))_{i\in\Pi}\leq y$.

Step 3: For each $i\in\Pi$, compute the values in $\mathcal{G}_{i}$.

Step 4: Verify that $\rho$ is the outcome of an NE (thanks to Theorem 4.1): as no new player reaches his target set along $\ell$ and $\ell$ is repeated infinitely often at the end of the outcome, we only have to check that $\rho$ is $\operatorname{\operatorname{Val}}$-consistent along $h\ell$ (an not along $h\ell^{\omega}$).

The second item is a direct consequence of Corollary 1.∎

To obtain the NP-completeness it remains to prove the NP-hardness.

Proposition 4

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game, for NE, Problem 1 is NP-hard.

This proof is based on a polynomial reduction from SAT and is inspired by the reduction provides for safety objective in [10].

Proof

To prove this proposition, we give a polynomial reduction from the SAT problem that is NP-complete. Let $X=\{x_{1},\ldots,x_{m}\}$ be the set of variables and $\psi=C_{1}\wedge\ldots\wedge C_{n}$ is a Boolean formula in CNF over $X$ and equals to the conjunction of the clauses $C_{1},\ldots,C_{n}$. This problem is to decide if the formula $\psi$ is true. Such a formula is true if there exists a valuation $\mathcal{I}:X\rightarrow\{0,1\}$ such that the valuation of $X$ with respect to $\mathcal{I}$ evaluates $\psi$ to true.

We build the following quantitative reachability game $\mathcal{G}_{\psi}=(\mathcal{A},(\operatorname{Cost}_{i})_{i\in\Pi},(F_{i})_{i\in\Pi})$ where $\mathcal{A}=(\Pi,V,E,(V_{i})_{i\in\Pi})$:

• •

the arena $\mathcal{A}$ is depicted in Figure 4 where the set of players $\Pi=\{1,\ldots,n,n+1\}$ has $n+1$ players: one by clause (players $1$ to $n$) and an additional one. $V_{n+1}$ is depicted by squared vertices and for all $1\leq i\leq n$, $V_{i}=\{P_{i}\}$;

• •

$F_{n+1}=\{T_{w}\}$;

• •

for all $1\leq i\leq n$, $F_{i}=\{0_{x}\mid\neg x\in C_{i}\}\cup\{1_{x}\mid x\in C_{i}\}\cup\{T_{\ell}\}.$

The game $\mathcal{G}_{\psi}$ can be build from $\psi$ in polynomial time. Let us show that $\psi$ is true if and only if there exists an NE $\sigma$ in $(\mathcal{G}_{\psi},x_{1})$ such that $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{x_{1}}))_{i\in\Pi}\leq(2m,\ldots,2m,2m+n)$.

$(\Rightarrow)$ Suppose that $\psi$ is true. Then there exists $\mathcal{I}:X\rightarrow\{0,1\}$ such that the valuation of $\psi$ with respect to $\mathcal{I}$ evaluates $\psi$ to true.

Let us consider $\sigma$ defined in this way:

• •

for all $hv\in\operatorname{Hist}_{n+1}(x_{1})$, $\sigma_{n+1}(hv)=\begin{cases}\mathcal{I}(v)&\text{ if }v=x\text{ with }x\in X\\ x_{k+1}&\text{ if }v=0_{x_{k}}\text{ or }1_{x_{k}}\text{with }0\leq k\leq m-1\\ P_{1}&\text{if }v=0_{x_{m}}\text{ or }1_{x_{m}}\\ T_{\ell}&\text{if }v=T_{\ell}\\ T_{w}&\text{if }v=T_{w}\end{cases}$.

• •

for all $1\leq i\leq n-1$ (resp. for $i=n$), for all $hv\in\operatorname{Hist}_{i}(x_{1})$ $\sigma_{i}(hv)=P_{i+1}$ (resp. $T_{w}$) if $hv$ is consistent with $\sigma_{n+1}$ and $\sigma_{i}(hv)=T_{\ell}$ otherwise.

We now prove that $\sigma$ is an NE. It is clear that $\langle\sigma\rangle_{x_{1}}$ is of the form $hP_{1}P_{2}\ldots P_{n}T_{w}^{\omega}$ and as $\sigma_{n+1}$ corresponds to the valuation $\mathcal{I}$ which evaluates $\psi$ to true, we have that : $\operatorname{Cost}_{n+1}(\langle\sigma\rangle_{x_{1}})=2m+n$ and for all $1\leq i\leq n$, $\operatorname{Cost}_{i}(\langle\sigma\rangle_{x_{1}})\leq 2m$. Obviously Player $n+1$ does not have an incentive to deviate from $\sigma_{n+1}$ because it is a the least cost that he can obtain. For each player $i$ such that $1\leq i\leq n$, as Player $i$ reaches his target set before he can play, changing his strategy does not change his cost. Thus, no player has a profitable deviation and $\sigma$ is an NE.

$(\Leftarrow)$ Suppose that there exists an NE $\sigma$ in $(\mathcal{G}_{\psi},x_{1})$ such that $\operatorname{Cost}_{i}(\langle\sigma\rangle_{x_{1}})\leq(2m,\ldots,2m,2m+n)$ and let us prove that $\psi$ is true.

We define $\mathcal{I}:X\rightarrow\{0,1\}$ as follows: for $x\in X$, $\mathcal{I}(x)=\sigma_{n+1}(hx)$ with $hx$ a prefix of $\langle\sigma\rangle_{x_{1}}$. Let us show that the valuation $\mathcal{I}$ evaluates $\psi$ to true. As $\operatorname{Cost}_{n+1}(\langle\sigma\rangle_{x_{1}})<+\infty$, it means that Player $n+1$ visits his target set $\{T_{w}\}$. Additionaly, for all $1\leq i\leq n$, as $\operatorname{Cost}_{i}(\langle\sigma\rangle_{x_{1}})<+\infty$, Player $i$ visits also his target set but not $T_{\ell}$. Thus, he reaches $\{0_{x}\mid\neg x\in C_{i}\}\cup\{1_{x}\mid x\in C_{i}\}$ and this means that the clause $C_{i}$ is true if each variable in $C_{i}$ is replaced by its valuation. As it is the case for each clause, the formula $\psi$ is true.

We conclude results about the threshold existence problem in quantitative reachability games with some remarks about a variant of this problem in this setting.

Remark 3

We may also consider this problem but with an upper and a lower threshold. This problem is also NP-complete but the result about memory is a little bit different. Indeed, the memory depends on the upper bound. If there exists an NE which satisfies the constraints then there exists an other one which satisfies the constraints too but with memory in $\mathcal{O}(\max_{i\in\Pi\mid y_{i}\neq+\infty}y_{i}+|V|+|\Pi|)$. The difference is due to the fact that we cannot apply iteratively the procedure (P1) and then procedure (P2) to obtain a lasso as procedure (P1) can decrease the cost of the outcome. So the new outcome could no longer satisfy the constraints. To solve this problem, we only have to apply procedure (P2) on the outcome of $\sigma$ and obtain a lasso of length at most $\max_{i\in\Pi\mid y_{i}\neq+\infty}y_{i}+|V|$.

Theorem 0.D.1

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game and $(\mathcal{X},x_{0})$ be its extended game,

• •

For SPE, (**[6]**) Problem 1 is PSPACE-complete.

• •

If the answer of the decision problem is positive, there exists a strategy profile $\sigma$ with memory in $\mathcal{O}(2^{|\Pi|}\cdot|\Pi|\cdot|V|^{(|\Pi|+1)\cdot(|\Pi|+|V|)+1})$ which satisfies the constraints and such that $\operatorname{Cost}_{i}(\langle\sigma\rangle_{x_{0}})\leq|V|\cdot|\Pi|$ if $i\in\operatorname{Visit}(\langle\sigma\rangle_{x_{0}})$.

Notice that, in [6], Problem 1 for SPE in quantitative reachability games is shown PSPACE-complete with a lower and an upper threshold.

Proof

Let us prove the second assertion. It is a direct consequence of Corollary 1. ∎

0.D.2 Proof of Theorem 4.3 and Theorem 4.4 for Problem 2

Lemma 5

Let $(\mathcal{G},v_{0})$ be a reachability game (resp.$(\mathcal{X},x_{0})$ its extended game) and let $w_{0}=v_{0}$ (resp. $w_{0}=x_{0}$) Let $k\in\{0,\ldots,|\Pi|\}$ and $c\in\mathbb{N}$ be two thresholds, let $p\in(\mathbb{N}\cup\{+\infty\})^{|\Pi|}$ be a cost profile and let $\sigma$ be a strategy profile in $(\mathcal{G},v_{0})$ (resp. $(\mathcal{X},x_{0})$). We define $\mathcal{R}=\{i\in\Pi\mid p_{i}<+\infty\}$. If $(|\mathcal{R}|,\sum_{i\in\mathcal{R}}p_{i})\succeq(k,c)$ and $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{w_{0}}))_{i\in\Pi}\leq p$ then $\operatorname{SW}(\langle\sigma\rangle_{w_{0}})\succeq(k,c)$.

Proof

Assume that $(|\mathcal{R}|,\sum_{i\in\mathcal{R}}p_{i})\succeq(k,c)$ and $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{w_{0}}))_{i\in\Pi}\leq p$. As $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{w_{0}}))_{i\in\Pi}\leq p$, we have that $|\operatorname{Visit}(\langle\sigma\rangle_{w_{0}})|\geq|\mathcal{R}|$.

Thus, $\operatorname{SW}(\langle\sigma\rangle_{w_{0}})\succeq(|\mathcal{R}|,\sum_{i\in\mathcal{R}}p_{i})$ and as the lexicographic ordering is transitive we have: $\operatorname{SW}(\langle\sigma\rangle_{v_{0}})\succeq(k,c)$. ∎

Proposition 5

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game.

• •

For NE, Problem 2 is NP-complete.

• •

If the answer to this decision problem is affirmative, then there exists a strategy profile $\sigma$ with memory in $\mathcal{O}((|\Pi|+1)\cdot|V|)$ which satisfies the constraints and such $\sum_{i\in\operatorname{Visit}(\langle\sigma\rangle_{v_{0}})}\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}})\leq|\Pi|^{2}\cdot|V|$ .

Proof

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game and let $k\in\{0,\ldots,|\Pi|\}$ and $c\in\mathbb{N}$ be two thresholds.

The NP-algorithm works as follows:

Step 1: Guess $p\in(\mathbb{N}\cup\{+\infty\})^{|\Pi|}$ a cost profile;

Step 2: Let $\mathcal{R}=\{i\in\Pi\mid p_{i}<+\infty\}$ be the set of players who have a finite cost in the cost profile $p$, verify if $(|\mathcal{R}|,\sum_{i\in\mathcal{R}}p_{i})\succeq(k,c)$;

Step 3: Verify that there exists an NE $\sigma$ in $(\mathcal{G},v_{0})$ such that $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}\leq p.(\star)$

Thanks to Proposition 3 and Lemma 5, this algorithm is an NP-algorithm to solve Problem 2 for Nash equilibrium. Moreover, if there exists an NE which fulfills constraints $(\star)$ there exists one with a polynomial memory and with costs less or equal to $|\Pi|\cdot|V|$ for players who have visited their target set (see Corollary 1). This leads to an NE such that the accumulated cost of the players who have visited their target set is less or equal to $|\Pi|^{2}\cdot|V|$ and it also fulfills the constraints for the social welfare.

The NP-hardness is due to a polynomial reduction from the SAT problem in the same philosophy than the one for Problem 1 for NE in quantitative reachability games: the SAT-formula is satisfiable if and only if the exists an NE with a social welfare $\succeq(|\Pi|,S)$ where $S=2mn+2m+n$ (sum of the components of the threshold fixed for the reduction for Problem 1). ∎

Theorem 0.D.2

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game and $(\mathcal{X},x_{0})$ be its extended game.

• •

For SPE, Problem 2 is PSPACE-complete.

• •

If the answer to this decision problem is positive, then there exists a strategy profile $\sigma$ with memory in $\mathcal{O}(2^{|\Pi|}\cdot|\Pi|\cdot|V|^{(|\Pi|+1)\cdot(|\Pi|+|V|)+1})$ which satisfies the constraints and such $\sum_{i\in\operatorname{Visit}(\langle\sigma\rangle_{x_{0}})}\operatorname{Cost}_{i}(\langle\sigma\rangle_{x_{0}})\leq|\Pi|^{2}\cdot|V|$ .

Proof

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game and $(\mathcal{X},x_{0})$ be its extended game and let $k\in\{0,\ldots,|\Pi|\}$ and $c\in\mathbb{N}$ be two thresholds.

The PSPACE-algorithm works as follows:

Step 1: Guess $p\in(\mathbb{N}\cup\{+\infty\})^{|\Pi|}$ a cost profile;

**Step 2:**Let $\mathcal{R}=\{i\in\Pi\mid p_{i}<+\infty\}$ be the set of players who have a finite cost in the cost profile $p$, verify if $(|\mathcal{R}|,\sum_{i\in\mathcal{R}}p_{i})\succeq(k,c)$;

Step 3: Verify that there exists an SPE $\sigma$ in $(\mathcal{X},x_{0})$ such that $(\operatorname{Cost}_{i}(\langle\sigma\rangle_{x_{0}}))_{i\in\Pi}\leq p.$ ($\star$)

Thanks to Theorem 0.D.1 and Lemma 5, this algorithm is a PSPACE-algorithm to solve Problem 2 for SPE. Moreover, if there exists an SPE which fulfills the constraints $(\star)$ there exists one with a finite-memory and with costs less or equal to $|\Pi|\cdot|V|$ for players who have visited their target set (see Corollary 1). This leads to an SPE such that the cost sum of the players who have visited their target set is less or equal to $|\Pi|^{2}\cdot|V|$ and it fulfills the constraints $(\star)$ and thus the constraints of Problem 2 (by Lemma 5).

The PSPACE-hardness is due to a polynomial reduction from the QBF problem (which is PSPACE-complete) in the same philosophy than the one for Problem 1 for SPE in quantitative reachability games (see [6]): the fully quantified Boolean formula is satisfiable if and only if the exists an SPE with a social welfare $\succeq(|\Pi|-1,S)$ where $S=2mn+2m+n$ (sum of the components of the threshold fixed for the reduction for Problem 1 for SPE in quantitative reachability games). But, if we want that the implication “$\Leftarrow$” holds, we have to slightly change the arena of the game (see Figure 5): we add a vertex $\perp$ that we add in the target set of the “existential player” (Player $n+1$) and that is reachable from $q_{1}\in V_{n+1}$ (we can assume that the formula $\psi=Q_{1}x_{1}Q_{2}x_{2}\ldots Q_{m}x_{m}\phi(X)$ is such that $Q_{k}=\exists$ if $k$ is odd and $Q_{k}=\forall$ otherwise). Notice that the weight $2S$ on the edge $(q_{1},\perp)$ should be understood as a path of length $2S$.

The main idea of the implication “$\Leftarrow$” is the following one: we have an SPE $\sigma$ sucht that $\operatorname{SW}(\langle\sigma\rangle_{q_{1}})\succeq(|\Pi|-1,S)$. As Player $n+1$ visits his target set if and only if Player $n+2$ does not, we have that $|\operatorname{Visit}(\langle\sigma\rangle_{q_{1}})|=|\Pi|-1$. Assume that Player $n+1$ does not visit his target set, as the game is initialized in $q_{1}$ which is a vertex of Player $n+1$, he can go to $\perp$ and in this way he visits his target set. So, it is a profitable deviation for Player $n+1$ and it leads to a contradiction with the fact that $\sigma$ is an SPE. We can conclude that $\sigma$ is an SPE such that all players excepted Player $n+2$ visit their target set.

Additionaly, for each $\rho\in\operatorname{Plays}(q_{1})$, if $\operatorname{Cost}_{n+1}(\rho)<+\infty$ (either the vertex $\perp$ or the vertex $t_{n+1}$ is reached) then no vertex $t_{k}$ with $1\leq k\leq n$ is reached. Then the philosophy of the proof is the same as the one in [6].

Notice that the weight on the edge $(q_{1},\perp)$ allows to ensure that, for implication “$\Rightarrow$”, Player $n+1$ does not have an incentive to go to $\perp$ in place to follow the valuation that makes the formula true.

∎

0.D.3 Proof of Theorem 4.3 and Theorem 4.4 for Problem 3

Lemma 6

Problem 4 belongs to co-NP for quantitative reachability games.

Proof

Let us prove it for quantitative reachability games. If $\rho$ is not Pareto optimal, there exists a play $\rho^{\prime}$ such that $(\operatorname{Cost}_{i}(\rho))_{i\in\Pi}\geq(\operatorname{Cost}_{i}(\rho^{\prime}))_{i\in\Pi}$ and $(\operatorname{Cost}_{i}(\rho))_{i\in\Pi}\neq(\operatorname{Cost}_{i}(\rho^{\prime}))_{i\in\Pi}$. Moreover, thanks to Lemma 3, one may assume that $\rho^{\prime}$ is a lasso with size at most $(|\Pi|+1)\cdot|V|$. So, we only have to guess such a lasso $\rho^{\prime}$ and to verify that $(\operatorname{Cost}_{i}(\rho))_{i\in\Pi}\geq(\operatorname{Cost}_{i}(\rho^{\prime}))_{i\in\Pi}$ and $(\operatorname{Cost}_{i}(\rho))_{i\in\Pi}\neq(\operatorname{Cost}_{i}(\rho^{\prime}))_{i\in\Pi}$. This can be done in polynomial time.

Proposition 6

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game, for NE:

• •

Problem 3 belongs to $\Sigma^{P}_{2}$ and is NP-hard.

• •

If the answer to this decision problem is positive, then there exists an NE $\sigma$ with memory in $\mathcal{O}((|\Pi|+1)\cdot|V|)$ such that for all $i\in\operatorname{Visit}(\langle\sigma\rangle_{v_{0}})$, $\operatorname{Cost}_{i}(\langle\sigma\rangle_{v_{0}})\leq|\Pi|\cdot|V|$.

Proof

We can provide the following $\Sigma^{P}_{2}$-algorithm for Problem 3, given an oracle for Problem 4:

Step 1: Guess a play $\rho$ as a lasso of length at most $(|\Pi|+1)\cdot|V|$ (sufficient thanks to Proposition 1).

Step 2: Check that $(\operatorname{Cost}_{i}(\rho))_{i\in\Pi}$ is Pareto optimal in $\operatorname{Plays}(v_{0})$ using the oracle for Problem 4.

Step 3: Check that $\rho$ is the outcome of a Nash equilibrium using the characterization.

Additionally, as $\rho$ is a lasso with length at most $(|\Pi|+1)\cdot|V|$, it provides an NE with outcome $\rho$ and memory in $\mathcal{O}((|\Pi|+1)\cdot|V|)$ such that for all $i\in\operatorname{Visit}(\rho)$, $\operatorname{Cost}_{i}(\rho)\leq|V|\cdot|\Pi|$ (by Proposition 1).

The NP-hardness is due to a polynomial reduction from the SAT problem which is NP-complete. The philosophy of the proof is the same than the one for the qualitative setting (see proof of Proposition 10) but we have to put a sufficiently high weight on the edge between the initial vertex and the vertex $\perp$ to ensure that Player $n+1$ does not have an incentive to go to $\perp$.∎

Proposition 7

Let $(\mathcal{G},v_{0})$ be a quantitative reachability game and let $(\mathcal{X},x_{0})$ its extended game, for SPE:

• •

Problem 3 is PSPACE-complete.

• •

If the answer to this decision problem is positive, then there exists an SPE $\sigma$ with memory with memory in $\mathcal{O}(2^{|\Pi|}\cdot|\Pi|\cdot|V|^{(|\Pi|+1)\cdot(|\Pi|+|V|)+1})$ which satisfies the constraints such that for all $i\in\operatorname{Visit}(\langle\sigma\rangle_{x_{0}})$, $\operatorname{Cost}_{i}(\langle\sigma\rangle_{x_{0}})\leq|\Pi|\cdot|V|$.

Proof

The PSPACE-algorithm works as follows

Step 1: Guess a cost profile $p\in(\mathbb{N}\cup\{+\infty\})^{|\Pi|}$;

Step 2: Check that $p$ is Pareto optimal in $\operatorname{Plays}(x_{0})$ using the oracle for Problem 4.

Step 3: Verify that there exists an SPE $\tau$ such that $p\leq(\operatorname{Cost}_{i}(\langle\tau\rangle_{x_{0}}))_{i\in\Pi}\leq p$ (Problem 1)

We obtain an SPE $\tau$ such that $(\operatorname{Cost}_{i}(\langle\tau\rangle_{x_{0}}))_{i\in\Pi}=p$ is Pareto optimal. By Proposition 1, we have that for all $i\in\operatorname{Visit}(\langle\tau\rangle_{x_{0}})$, $\operatorname{Cost}_{i}(\langle\tau\rangle_{x_{0}})\leq|V|\cdot|\Pi|$. Thus, by Proposition 2, there exists an SPE $\sigma$ with the same cost profile than $\tau$ but with memory in $\mathcal{O}(2^{|\Pi|}\cdot|\Pi|\cdot|V|^{(|\Pi|+1)\cdot(|\Pi|+|V|)+1})$.

The PSPACE-hardness is due to a polynomial reduction from the QBF problem which is PSPACE-complete. The philosophy of the proof is the same than the one for the qualitative setting (see proof of Proposition 13) but we have to put a sufficiently high weight on the edge between the initial vertex and the vertex $\perp$.∎

Appendix 0.E Qualitative reachability games

0.E.1 Additional preliminaries

0.E.1.1 Qualitative reachability games

All along this section we focus on qualitative reachability games. Unlike quantitative reachability games, the arena is equipped with a gain function profile $(\operatorname{Gain}_{i})_{i\in\Pi}$ such that for all $i\in\Pi$, $\operatorname{Gain}_{i}:\operatorname{Plays}\rightarrow\mathbb{N}\cup\{+\infty\}$ is a gain function which assigns a gain to each play $\rho$ for Player $i$. We also say that the play $\rho$ has gain profile $(\operatorname{Gain}_{i}(\rho))_{i\in\Pi}$ and similarly if we consider the outcome of the strategy profile $\sigma$ from $v_{0}$, we say that $\sigma$ has gain profile $(\operatorname{Gain}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}$.

Definition 11

A qualitative reachability game $\mathcal{G}=(\mathcal{A},(\operatorname{Gain}_{i})_{i\in\Pi},(F_{i})_{i\in\Pi})$ is a game enhanced with a target set $F_{i}\subseteq V$. For all $i\in\Pi$, the gain function $\operatorname{f}_{i}=\operatorname{Gain}_{i}$ is defined as follows: for all $\rho=\rho_{0}\rho_{1}\ldots\in\operatorname{Plays}$: $\operatorname{Gain}_{i}(\rho)=1$ if there exists $k\in\mathbb{N}$ such that $\rho_{k}\in F_{i}$ and $\operatorname{Gain}_{i}(\rho)=0$ otherwise.

In this particular setting, players only aim to reach their target set but do not take into account the number of steps it takes. Player $i$ receives a gain of $1$ if $\rho$ visits his target set $F_{i}$, and a gain of [math] otherwise. Thus each player $i$ wants to maximize his gain.

0.E.1.2 Solution concepts

For qualitative reachability games, it is easy to recover the definitions of NE and SPE defined in Section 2.0.3 by reversing the inequality and replacing cost functions by gain functions, as players want to maximize their gain instead of minimizing their cost. This leads to the following Lemma.

Lemma 7

Let $(\mathcal{G},v_{0})$ be an initialized quantitative reachability game and $\sigma$ be a strategy profile. Consider the related qualitative reachability game $\mathcal{G}^{\prime}$ with the same arena $\mathcal{A}$ and target sets $(F_{i})_{i\in\Pi}$, but the gain functions $(\operatorname{Gain}_{i})_{i\in\Pi}$. Then if $\sigma$ is an NE (resp. SPE) in $(\mathcal{G},v_{0})$, then $\sigma$ is also an NE (resp. SPE) in $(\mathcal{G}^{\prime},v_{0})$.

Thus, as it is proved that there always exists an SPE (and thus an NE) in a quantitative reachability game, there always exists one in a qualitative reachability game.

Theorem 0.E.1

In every initialized qualitative reachability game, there always exists an SPE, and thus also an NE.

0.E.1.3 Studied problems

In case of qualitative reachability, as for quantitative reachability game we are interested in a solution that fulfills certain requirements. For example, we would like to know whether there exists a solution such that a maximum number of players visit their target sets.

Let $(\mathcal{G},v_{0})$ be an initialized qualitative reachability game with $\mathcal{G}=$

$(\mathcal{A},(\operatorname{Gain}_{i})_{i\in\Pi},(F_{i})_{i\in\Pi})$. Given $\rho\in\operatorname{Plays}(v_{0})$, we denote by $\operatorname{Visit}(\rho)$ the set of players $i$ such that $\rho$ visits $F_{i}$, that is, $\operatorname{Visit}(\rho)=\{i\in\Pi\mid Gain_{i}(\rho)=1\}$. The social welfare $\operatorname{SW}(\rho)$ of $\rho$ is the size of $\operatorname{Visit}(\rho)$. Let $P\subseteq\{0,1\}^{|\Pi|}$ be the set of all gain profiles $p=(\operatorname{Gain}_{i}(\rho))_{i\in\Pi}$, with $\rho\in\operatorname{Plays}(v_{0})$. A cost profile $p\in P$ is called Pareto-optimal in $\operatorname{Plays}(v_{0})$ if it is maximal in $P$ with respect to the componentwise ordering $\leq$ on $P$. Notice that if there exists $\rho$ with $\operatorname{Visit}(\rho)=\Pi$, then its social welfare is the largest possible and there exists a unique Pareto optimal gain profile equal to $(1,1,\ldots,1)$. Notice also that certain target sets $F_{i}$ might be empty or not reachable from the initial vertex $v_{0}$. Hence in this case, the best that we can hope is a (unique) Pareto optimal gain profile $p$ such that $p_{i}=1$ if and only if $F_{i}$ is reachable666Notice that if $F_{i}$ is reachable from $v_{0}$, then it is necessarily not empty. from $v_{0}$.

Problem 5 (Threshold decision problem)

Given an initialized qualitative reachability game $(\mathcal{G},v_{0})$, given two thresholds $x,y\in\{0,1\}^{|\Pi|}$, decide whether there exists a solution $\sigma$ such that $x\leq(\operatorname{Gain}_{i}(\rho))_{i\in\Pi}\leq y$.

Imposing a lower bound $x_{i}=1$ means that player $i$ has to visit his target set whereas imposing an upper bound $y_{i}=0$ means that player $i$ cannot visit his target set.

Unlike quantitative reachability, social welfare in qualitative reachability games only aims to maximize the number of players who visit their target set.

Problem 6 (Social welfare decision problem)

Given an initialized qualitative reachability game $(\mathcal{G},v_{0})$, given a threshold $k\in\{0,\ldots,|\Pi|\}$, decide whether there exists a solution $\sigma$ such that $\operatorname{SW}(\langle\sigma\rangle_{v_{0}})\geq k$.

Let us now state the last studied problem for qualitative reachability games.

Problem 7 (Pareto-optimal decision problem)

Given an initialized qualitative reachability game $(\mathcal{G},v_{0})$ decide whether there exists a solution $\sigma$ in $(\mathcal{G},v_{0})$ such that $(\operatorname{Gain}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}$ is Pareto optimal in $\operatorname{Plays}(v_{0})$.

This problem has some connections with the two previous ones. For instance in case of qualitative reachability, suppose there exists a play in $\operatorname{Plays}(v_{0})$ that visits all target sets. As already explained, there is only one Pareto-optimal gain $(1,\ldots,1)$. Asking for the existence of a solution $\sigma$ such that $(\operatorname{Gain}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}$ is Pareto-optimal is equivalent to asking for the existence of a solution $\sigma$ such that $\operatorname{Gain}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}\geq(1,\ldots,1)$ (see Problem 5), or such that $\operatorname{SW}(\langle\sigma\rangle_{v_{0}})\geq|\Pi|$ (see Problem 6).

0.E.2 Existence problem

The following Theorem is a direct consequence of Theorem 3.1 and Lemma 7.

Theorem 0.E.2

Let $(\mathcal{G},v_{0})$ be an initialized qualitative reachability game such that its arena $\mathcal{A}$ is strongly connected. Then there exists an SPE $\sigma$ (and thus an NE) such that its outcome $\langle\sigma\rangle_{v_{0}}$ visits all target sets $F_{i}$, $i\in\Pi$, that are non empty.

Let us comment this result. For this family of games, the answer to Problem 5 is always positive for particular thresholds. Take thresholds $x,y$ such that $x_{i}=1$ (and thus $y_{i}=1$) if and only if $F_{i}\neq\emptyset$. The answer to Problem 6 is also always positive for threshold $k=|\{i\mid F_{i}\neq\emptyset\}|$. Finally, the answer to Problem 3 is also always positive since there exists an unique Pareto optimal gain profile $p$ such that $p_{i}=1$ if and only if $F_{i}\neq\emptyset$.

Recall that we explained before why it was enough to prove Theorem 3.1 for SPEs and for quantitative reachability games only. Notice that in case of qualitative reachability games, there exists a simpler construction of the required NE or SPE. Indeed, as the arena is strongly connected, there exists a play $\rho\in\operatorname{Plays}(v_{0})$ that visits all non empty target sets. (i) Hence to get an NE, construct a strategy profile $\sigma$ in $(\mathcal{G},v_{0})$ such that $\langle\sigma\rangle_{v_{0}}=\rho$. As the gain profile of $\sigma$ is the best that each player can hope, no player has an incentive to deviate and $\sigma$ is then an NE. (ii) The construction is a little more complex to get an SPE. We again construct a strategy profile $\sigma$ in $(\mathcal{G},v_{0})$ such that $\langle\sigma\rangle_{v_{0}}=\rho$, and inductively extend its construction to all subgames $(\mathcal{G}_{\restriction h},v)$ as follows. Assume that $\sigma_{\restriction h}$ is not yet constructed, then extend the construction of $\sigma$ such that $\sigma_{\restriction h}=g\rho$ for some $gv_{0}$ starting in $v$ and ending in $v_{0}$ (such a history $gv_{0}$ exists because the arena is strongly connected). In this way, the outcome of $\sigma_{\restriction h}$ in each subgame $(\mathcal{G},v)$ has gain profile $(1,\ldots,1)$ and no player has an incentive to deviate. It follows that $\sigma$ is an SPE.

The next theorem states that Problem 3 has a positive answer for all qualitative reachability games with a number of players limited to two, and that this existence result cannot be extended to three players.

Theorem 0.E.3

Let $(\mathcal{G},v_{0})$ be an initialized qualitative reachability game,

• •

Let $(\mathcal{G},v_{0})$ be an initialized qualitative reachability game such that $|\Pi|=2$, there exists an SPE $\sigma$ (and thus an NE) with a gain profile that is Pareto optimal in $\operatorname{Plays}(v_{0})$.

• •

There exists an initialized qualitative reachability games with $|\Pi|=3$ that has no NE with a gain profile that is Pareto optimal in $\operatorname{Plays}(v_{0})$.

Let us focus on the proof of Theorem 0.E.3 which is based on the next lemma, which is interesting in its own right.

Lemma 8

Let $(\mathcal{G},v_{0})$ be an initialized qualitative reachability game. Let $p$ be a gain profile equal to $(0,0,\ldots,0)$ or $(1,1,\ldots,1)$. If $p$ is Pareto-optimal777$(1,1,\ldots,1)$ is trivially Pareto-optimal. in $\operatorname{Plays}(v_{0})$, then there exists an SPE $\sigma$ with gain profile $p$.

Proof

The case $p=(0,0,\ldots,0)$ is easy to solve. By Pareto-optimality, all plays in $\operatorname{Plays}(v_{0})$ have gain profile $p$. Hence every strategy profile $\sigma$ is trivially an SPE with gain profile $p$. Let us turn to case $p=(1,1,\ldots,1)$ and let $\rho=\rho_{0}\rho_{1}\ldots\in\operatorname{Plays}(v_{0})$ with gain profile $p$. By Theorem 2.1 in [3]888Notice that we cannot apply Theorem 3.1 since the arena is not necessarily strongly connected., there exists an SPE $\sigma$ in $(\mathcal{G},v_{0})$. If $(\operatorname{Gain}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}=p$, we are done. Otherwise let us show how to modify $\sigma$ into another SPE $\tau$ with outcome $\rho$ and thus with gain profile $p$. Let $h\in\operatorname{Hist}_{i}(v_{0})$, $i\in\Pi$,

• •

if $h$ is a prefix of $\rho$, then $\tau_{i}(h)=\rho_{|h|+1}$,

• •

otherwise, $\tau_{i}(h)=\sigma_{i}(h)$.

Let us prove that $\tau$ is an SPE. Clearly for each history $hv$ that is not a prefix of $\rho$, $\tau_{\restriction h}=\sigma_{\restriction h}$ is an NE in the subgame $(\mathcal{G}_{\restriction h},v)$. So let $hv=\rho_{0}\ldots\rho_{k}$. As $\langle\tau_{\restriction h}\rangle_{v}$ has gain profile $(1,1,\ldots,1)$ in $(\mathcal{G}_{\restriction h},v)$, player $i$ such that $v\in V_{i}$ has no incentive to deviate, and then $\tau_{\restriction h}$ is also an NE in $(\mathcal{G}_{\restriction h},v)$. ∎

Proof (of Theorem 0.E.3)

We begin with the first item. There are three cases to study: either the unique Pareto-optimal gain profile of $\operatorname{Plays}(v_{0})$ is equal to $(0,0)$, or it is equal to $(1,1)$, or there are one or two Pareto-optimal gain profiles that belong to $\{(0,1),(1,0)\}$. In the first two cases, we get the required SPE by Lemma 8. Hence it remains to treat the last case. From Lemma 2, we know that there exists an SPE in $(\mathcal{G},v_{0})$ whose outcome $\rho$ visits a least one target set $F_{i}$, $i\in\{1,2\}$. Therefore the gain profile of $\rho$ is either equal to $(0,1)$ or $(1,0)$ as required.

For the second item, consider the initialized qualitative reachability game $(\mathcal{G},v_{0})$ of Figure 6. We have three players such that player $3$ owns diamond vertices. Moreover, $F_{1}=\{v_{4},v_{5}\}$, $F_{2}=\{v_{3},v_{5}\}$, and $F_{3}=\{v_{4},v_{6}\}$. There are four plays in $\operatorname{Plays}(v_{0})$ whose gain profile is indicated below each of them. The set of Pareto-optimal gain profiles in $\operatorname{Plays}(v_{0})$ is equal to $\{(1,0,1),(1,1,0)\}$. Consider a strategy profile $\sigma$ with outcome $v_{0}v_{1}v_{4}^{\omega}$ and gain profile $(1,0,1)$. Then it is not an NE because player $2$ has a profitable deviation by going from $v_{1}$ to $v_{3}$ (instead of $v_{4}$). Similarly the strategy profile $\sigma$ with outcome $v_{0}v_{2}v_{5}^{\omega}$ and gain profile $(1,1,0)$ is not an NE. Therefore there is no NE in $(\mathcal{G},v_{0})$ with a gain profile that is Pareto-optimal.

∎

0.E.3 Decision problems

0.E.3.1 $\lambda$-consistent play

The notion of $\lambda$-consistent play explained in Section 4.1 can easily be adapted for qualitative reachability games. In the qualitative setting, $\lambda(v)$ allows us to know if the player who owns $v$ can ensure to reach his target set from $v$ or not.

0.E.3.2 Lassoes with polynomial size

Lemma 9

Let $(\mathcal{G},v_{0})$ be a qualitative reachability game and $\rho\in\operatorname{Plays}$ be a play.

• •

If $\rho^{\prime}$ is the play obtained by applying (P1) or (P2) on $\rho$, then for all $i\in\Pi$, $\operatorname{Gain}_{i}(\rho^{\prime})=\operatorname{Gain}_{i}(\rho)$.

• •

Applying (P1) until it is no longer possible and then (P2), leads to a lasso $\rho^{\prime}$ with size at most $(|\Pi|+1)\cdot|V|$.

And the property on $\lambda$-consistence after applying (P1) or (P2) also holds.

Lemma 10

Let $(\mathcal{G},v_{0})$ be a qualitative reachability game and $\rho\in\operatorname{Plays}$ be a $\lambda$-consistent play for a given labeling function $\lambda$. If $\rho^{\prime}$ is the play obtained by applying (P1) or (P2) on $\rho$, then $\rho^{\prime}$ is $\lambda$-consistent.

0.E.3.3 Results on Nash equilibria

Characterization of outcomes of NEs

The characterization provides in Section 4.2 for NEs remains true for qualitative reachability games with some modifications: using the notion of qualitative coalitional game in place of quantitative coalitional game, replacing $\operatorname{Cost}$ by $\operatorname{Gain}$ in the characterization, reversing the inequalities, $\ldots$

Notions of coalitional game, value and optimal strategies can be rewritten if $\mathcal{G}$ is a qualitative reachability game and we obtain $|\Pi|$ (qualitative) coalitional games. Qualitative coalitional games are also determined, there exist positional optimal strategies for both players and the values can be computed in polynomial time [17].

Complexity and memory results

Theorem 0.E.4

Let $(\mathcal{G},v_{0})$ be a qualitative reachability game, for NE: Problem 5 and Problem 6 are NP-complete while Problem 7 is NP-hard and belongs to $\Sigma^{P}_{2}$.

Theorem 0.E.5

Let $(\mathcal{G},v_{0})$ be a qualitative reachability game, for NE: for each decision problem, if its answer is positive, then there exists a strategy profile $\sigma$ with memory in $\mathcal{O}((|\Pi|+1)\cdot|V|)$ which satisfies the conditions.

These two theorems are due to Propositions 8,9 and 10

Proposition 8 ([10])

Let $(\mathcal{G},v_{0})$ be a qualitative reachability game, for NE:

• •

Problem 5 is NP-complete.

• •

If the answer to this decision problem is positive, then there exists a strategy profile with memory in $\mathcal{O}((|\Pi|+1)\cdot|V|)$ which satisfies the constraints.

Proposition 9

Let $(\mathcal{G},v_{0})$ be a qualitative reachability game.

• •

For NE, Problem 6 is NP-complete.

• •

If the answer to this decision problem is positive, then there exists a strategy profile with memory in $\mathcal{O}((|\Pi|+1)\cdot|V|)$ which satisfies the constraints.

Proof

Let $(\mathcal{G},v_{0})$ be a qualitative reachability game and let $k\in\{0,\ldots,|\Pi|\}$ be a threshold.

The NP-algorithm works as follows:

Step 1: Guess $p\in\{0,1\}^{|\Pi|}$ a gain profile (sufficient thanks to Lemma 9);

Step 2: Let $\mathcal{R}=\{i\in\Pi\mid p_{i}=1\}$ be the set of players who have a gain equal to 1 in the gain profile $p$, verify if $|\mathcal{R}|\geq k$;

Step 3: Verify that there exists an NE $\sigma$ in $(\mathcal{G},v_{0})$ such that $(\operatorname{Gain}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}\geq p.$

As if $(\operatorname{Gain}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}\geq p$ then $|\operatorname{Visit}(\langle\sigma\rangle_{v_{0}})|\geq|\mathcal{R}|\geq k$, and thanks to Proposition 8, this algorithm is an NP-algorithm to solve Problem 6 for Nash equilibrium. Moreover, if there exists an NE which fulfills the constraints there exists one with a polynomial memory which fulfills the constraints too.

The NP-hardness is due to a polynomial reduction from the SAT problem in the same philosophy than the one for Problem 1 (see Proposition 4) for NE in quantitative reachability games: the SAT-formula is satisfiable if and only if there exists an NE with a social welfare $\geq|\Pi|$. ∎

Proposition 10

Let $(\mathcal{G},v_{0})$ be a qualitative reachability game, for NE:

• •

Problem 7 belongs to $\Sigma^{P}_{2}$ and is NP-hard.

• •

If the answer of this decision problem is positive, then there exists a strategy profile with memory in $\mathcal{O}((|\Pi|+1)\cdot|V|)$ which satisfies the constraints.

The proof of Proposition 10 relies on the complexity of the variant of Problem 4 for qualitative reachability games.

Problem 8

Given a qualitative reachability game $(\mathcal{G},v_{0})$ and a lasso $\rho\in\operatorname{Plays}$, we want to decide if $(\operatorname{Gain}_{i}(\rho))_{i\in\Pi}$ is Pareto optimal in $\operatorname{Plays}(v_{0})$.

This decision problem belongs to co-NP.

Lemma 11

Problem 8 belongs to co-NP for qualitative reachability games.

Proof

The same proof as for quantitative reachability games holds by using $\operatorname{Gain}$ in place of $\operatorname{Cost}$ and reversing the inequalities. ∎

Proof (of Proposition 10)

We can provide the following $\Sigma^{P}_{2}$-algorithm for Problem 3, given an oracle for Problem 8:

Step 1: Guess a play $\rho$ as a lasso of size $(|\Pi|+1)\cdot|V|$ (sufficient thanks to Lemma 9);

Step 2: Check that $(\operatorname{Gain}_{i}(\rho))_{i\in\Pi}$ is Pareto optimal in $\operatorname{Plays}(v_{0})$ using the oracle for Problem 8.

Step 3: Check that $\rho$ is the outcome of a Nash equilibrium using the characterization.

The NP-hardness is due to a polynomial reduction from the SAT problem that is NP-complete. Let $X=\{x_{1},\ldots,x_{m}\}$ be the set of variables and $\psi=C_{1}\wedge\ldots\wedge C_{n}$ is a Boolean formula in CNF over $X$ and equals to the conjunction of the clauses $C_{1},\ldots,C_{n}$. This problem is to decide if the formula $\psi$ is true. Such a formula is true if there exists a valuation $\mathcal{I}:X\rightarrow\{0,1\}$ such that the valuation of $X$ with respect to $\mathcal{I}$ evaluates $\psi$ to true.

We build the following qualitative reachability game $\mathcal{G}_{\psi}=(\mathcal{A},(\operatorname{Gain}_{i})_{i\in\Pi},(F_{i})_{i\in\Pi})$ where $\mathcal{A}=(\Pi,V,E,(V_{i})_{i\in\Pi})$:

• •

the arena $\mathcal{A}$ is depicted in Figure 7 where the set of players $\Pi=\{1,\ldots,n,n+1\}$ has $n+1$ players: one by clause (players $1$ to $n$) and an additional one. $V_{n+1}$ is depicted by squared vertices and for all $1\leq i\leq n$, $V_{i}=\{P_{i}\}$;

• •

$F_{n+1}=\{T_{w},\perp\}$;

• •

for all $1\leq i\leq n$, $F_{i}=\{0_{x}\mid\neg x\in C_{i}\}\cup\{1_{x}\mid x\in C_{i}\}\cup\{T_{\ell}\}.$

The game $\mathcal{G}_{\psi}$ can be build from $\psi$ in polynomial time. Let us show that $\psi$ is true if and only if there exists an NE $\sigma$ in $(\mathcal{G}_{\psi},x_{1})$ such that $\operatorname{Gain}(\langle\sigma\rangle_{x_{1}})$ is Pareto optimal in $\operatorname{Plays}(x_{1})$.

$(\Rightarrow)$ Suppose that $\psi$ is true. Then there exists $\mathcal{I}:X\rightarrow\{0,1\}$ such that the valuation of $\psi$ with respect to $\mathcal{I}$ evaluates $\psi$ to true.

Let us consider $\sigma$ defined in this way:

• •

for all $hv\in\operatorname{Hist}_{n+1}(x_{1})$, $\sigma_{n+1}(hv)=\begin{cases}\mathcal{I}(v)&\text{ if }v=x\text{ with }x\in X\\ x_{k+1}&\text{ if }v=0_{x_{k}}\text{ or }1_{x_{k}}\text{with }0\leq k\leq m-1\\ P_{1}&\text{if }v=0_{x_{m}}\text{ or }1_{x_{m}}\\ T_{\ell}&\text{if }v=T_{\ell}\\ T_{w}&\text{if }v=T_{w}\\ \perp&\text{if }v=\perp\end{cases}$.

• •

for all $1\leq i\leq n-1$ (resp. for $i=n$), for all $hv\in\operatorname{Hist}_{i}(x_{1})$ $\sigma_{i}(hv)=P_{i+1}$ (resp. $T_{w}$) if $hv$ is consistent with $\sigma_{n+1}$ and $\sigma_{i}(hv)=T_{\ell}$ otherwise.

We now prove that $\sigma$ is an NE. It is clear that $\langle\sigma\rangle_{x_{1}}$ is of the forme $hP_{1}P_{2}\ldots P_{n}T_{w}^{\omega}$ and as $\sigma_{n+1}$ corresponds to the valuation $\mathcal{I}$ which evaluates $\psi$ to true, we have that for all $i\in\Pi$, $\operatorname{Gain}_{i}(\langle\sigma\rangle_{x_{1}})=1$. Obviously no player has a profitable deviation and $(\operatorname{Gain}_{i}(\langle\sigma\rangle_{x_{1}}))_{i\in\Pi}$.

$(\Leftarrow)$ We will prove its contrapositive. Assume that $\psi$ is not true. As $\psi$ is not true, there exists a player $i\in\Pi$ such that if he wants to win (obtain a gain of 1) he has to go to $T_{\ell}$. However, in this way Player $n+1$ does not reach his target set and his only possibility to do so is to go to $\perp$. It means that the only possible outcome for an NE is $x_{1}\perp^{\omega}$ with gain profile $(0,\ldots,0,1)$ which is not Pareto optimal in $\operatorname{Plays}(x_{1})$. Indeed, as at least one clause can be evaluated to true, assume it is clause $C_{i}$ (with $1\leq i\leq n$), there exists a play in $\operatorname{Plays}(x_{1})$ such that Player $i$ visits his target set and which ends in $T_{w}$.

∎

0.E.3.4 Results on subgame perfect equilibria

Complexity and memory results

Theorem 0.E.6

Let $(\mathcal{G},v_{0})$ be a qualitative reachability game, for SPE: Problem 5, Problem 6 and Problem 7 are PSPACE-complete.

Theorem 0.E.7

Let $(\mathcal{G},v_{0})$ be a qualitative reachability game, for SPE: for each decision problem, if the answer is positive, there exists a strategy profile $\sigma$ with memory in $\mathcal{O}(|V|^{3}\cdot|\Pi|\cdot 2^{3\cdot|\Pi|})$ which satisfies the conditions.

These two theorems are due to Propositions 11,12 and 13.

Proposition 11 ([4])

Let $(\mathcal{G},v_{0})$ be a qualitative reachability game.

• •

For SPE, Problem 5 is PSPACE-complete.

• •

If the answer of the decision problem is positive, there exists a strategy profile with memory in $\mathcal{O}(|V|^{3}\cdot|\Pi|\cdot 2^{3\cdot|\Pi|})$ which satisfies the constraints.

Proposition 12

Let $(\mathcal{G},v_{0})$ be a qualitative reachability game.

• •

For SPE, Problem 6 is PSPACE-complete.

• •

If the answer of the decision problem is positive, there exists a strategy profile with memory in $\mathcal{O}(|V|^{3}\cdot|\Pi|\cdot 2^{3\cdot|\Pi|})$ which satisfies the constraints.

Proof

Let $(\mathcal{G},v_{0})$ be a qualitative reachability game and let $k\in\{0,\ldots,|\Pi|\}$ be a threshold.

The PSPACE-algorithm works as follows:

Step 1: Guess $p\in\{0,1\}^{|\Pi|}$ a gain profile;

Step 2: Let $\mathcal{R}=\{i\in\Pi\mid p_{i}=1\}$ be the set of players who have a cost equal to 1 in the cost profile $p$, verify if $|\mathcal{R}|\geq k$;

Step 3: Verify that there exists an SPE $\sigma$ in $(\mathcal{G},v_{0})$ such that $(\operatorname{Gain}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}\geq p.$

As if $(\operatorname{Gain}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}\geq p$ then $|\operatorname{Visit}(\langle\sigma\rangle_{v_{0}})|\geq|\mathcal{R}|\geq k$, and thanks to Proposition 11, this algorithm is a PSPACE-algorithm to solve Problem 6 for SPE. Moreover, if there exists an SPE which fulfills the constraints there exists one with a finite memory which fulfills the constraints too.

The PSPACE-hardness is due to a polynomial reduction from the QBF problem (which is PSPACE-complete) in the same philosophy than the one for Problem 2 for SPE in quantitative reachability games: the fully quantified formula is satisfiable if and only if the exists an SPE with a social welfare $\geq|\Pi|-1$. Notice that in the qualitative setting, there is no weight on the edge $(q_{1},\perp)$ in the arena depicted in Figure 5. ∎

Proposition 13

Let $(\mathcal{G},v_{0})$ be a qualitative reachability game,

• •

For SPE, Problem 7 is PSPACE-complete.

• •

If the answer of the decision problem is positive, there exists a strategy profile with memory in $\mathcal{O}(|V|^{3}\cdot|\Pi|\cdot 2^{3\cdot|\Pi|})$ which satisfies the constraints.

Proof

We can provide the following PSPACE-algorithm for Problem 7, given an oracle for Problem 8:

Step 1: Guess a payoff $p\in\{0,1\}^{|\Pi|}$.

Step 2: Check that $p$ is Pareto optimal in $\operatorname{Plays}(v_{0})$ using the oracle for Problem 8.

Step 3: Check if there exists an SPE $\sigma$, such that $p\leq(\operatorname{Gain}_{i}(\langle\sigma\rangle_{v_{0}}))_{i\in\Pi}\leq p$.

By Lemma 11 and Proposition 11, Step 2 is done in co-NP and Step 3 is done in PSPACE. It leads to a PSPACE-algorithm.

The PSPACE-hardness is due to a polynomial reduction from the QBF problem (which is PSPACE-complete) in the same philosophy than the one for Problem 2 for SPE in quantitative reachability games: the fully quantified formula is satisfiable if and only if the exists an SPE with a gain profile which is Pareto optimal. Notice that in the qualitative setting, there is no weight on the edge $(q_{1},\perp)$ in the arena depicted in Figure 5.

For the implication “$\Leftarrow$”, we assume that we have an SPE $\sigma$ such that $(\operatorname{Gain}_{i}(\langle\sigma\rangle_{q_{1}}))_{i\in\Pi}$ is Pareto optimal in $\operatorname{Plays}(q_{1})$ and we want to prove that the fully quantified formula $\psi$ is satisfied. In the game $\mathcal{G}_{\psi}$, there is only two Pareto optimal gain profile: $(g_{1},\ldots,g_{n},1,0)$ and $(g_{1},\ldots,g_{n},0,1)$ where $g_{i}=1$ if the clause $C_{i}$ is satisfiable in $\psi$ and $g_{i}=0$ otherwise. One can prove that the gain profile $(g_{1},\ldots,g_{n},0,1)$ cannot be the gain profile of an SPE as in this gain profile Player $n+1$ has a gain of [math] and Player $n+1$ can chose to go in $\perp$ to ensure a gain of $1$. Thus, we know that $\sigma$ is an SPE with gain profile $(g_{1},\ldots,g_{n},1,0)$ and in particular such that Player $n+1$ visits his target set. Then, the same kind of arguments used in the proof of Problem 1 (see [6]) can be used. ∎