Reachability Games in Dynamic Epistemic Logic

TL;DR

This paper introduces reachability games within Dynamic Epistemic Logic, analyzing strategies for controllers and agents with perfect and imperfect information, and provides decidability results for various action types.

Contribution

It formalizes reachability games in DEL, extending epistemic planning to multi-agent settings with new decidability and undecidability results.

Findings

Decidability results for controller strategies with public actions

Undecidability results for distributed strategies among agents

Extension of epistemic planning to multi-agent, imperfect information scenarios

Abstract

We define reachability games based on Dynamic Epistemic Logic (DEL), where the players' actions are finely described as DEL action models. We first consider the setting where an external controller with perfect information interacts with an environment and aims at reaching some epistemic goal state regarding the passive agents of the system. We study the problem of strategy existence for the controller, which generalises the classic epistemic planning problem, and we solve it for several types of actions such as public announcements and public actions. We then consider a yet richer setting where agents themselves are players, whose strategies must be based on their observations. We establish several (un)decidability results for the problem of existence of a distributed strategy, depending on the type of actions the players can use, and relate them to results from the literature on multiplayer games with imperfect information.

Tables

Table 1. Table 1 : Known results (new in grey) for plan, controller and distributed strategy synthesis.

	Public announcements	Public actions	Propositional actions	Full
Plan	NP-c	PSPACE-c	decidable	undecidable
Controller	PSPACE-c (Th. 3.3)	EXPTIME-c (Th. 3.5)	decidable (Th. 3.7)	undecidable
			undecidable (Th. 4.3)
Distributed strategy	PSPACE-c (Th. 4.5)	EXPTIME-c (Th. 4.7)	decidable case (Th. 4.9)	undecidable

Equations

$$\phi ::=p\mid \neg \phi \mid (\phi \vee \phi )\mid K_a\phi$$

$$\mathcal{M}\otimes {\mathcal{A}}^n,(w,{\alpha }_1,\dots ,{\alpha }_n)\models \phi \text{mbox}iff\mathcal{M}{\mathcal{A}}^{\ast },w{\alpha }_1\dots {\alpha }_n\models \phi .$$

$$\exists p_1\forall p_2\dots \exists p_{2k-1}\forall p_{2k}\chi (p_1,\dots ,p_{2k})$$

$${\phi }_{\neg p_i}=\underset{j=1}{\overset{i-1}{\bigwedge }}{K}_a\neg q_j\wedge \underset{j=i}{\overset{2k}{\bigwedge }}{\hat{K}}_a{q}_j\wedge \neg p_i\wedge \neg q_i\text{mbox}and{\phi }_{p_i}=\underset{j=1}{\overset{i-1}{\bigwedge }}{K}_a\neg q_j\wedge \underset{j=i}{\overset{2k}{\bigwedge }}{\hat{K}}_a{q}_j\wedge \neg q_i$$

$$\scriptsize\begin{array}[]{l}pre:\varphi\\ post:action:=\langle\varphi,\overrightarrow{\text{post}}\rangle\end{array}$$

$$\scriptsize\begin{array}[]{l}pre:action=\langle\varphi,\overrightarrow{\text{post}}\rangle\\ post:\text{post}_{i}\end{array}$$

Full text

1 Introduction

Many applications fall within the scope of reachability games with imperfect information, such as video games [15] (Civilization, etc.), Kriegspiel (the epistemic variant of Chess) [25], Hanabi [3], or contingent and conformant planning [20].

Games with imperfect information are computationally hard, and even undecidable for multiple players [29]. One way to tame this complexity is to make assumptions on how the knowledge of the different players compare: if all players that cooperate can be ordered in a hierarchy where one knows more than the next, a situation called hierarchical information, then the existence of distributed strategies can be decided [28, 7]. Another natural approach is to consider fragments based on classes of action types, as done for instance in [32, 6, 11] where different kinds of public actions are considered. But the usual graph-based models of games with imperfect information, where the players’ actions are modelled as labels on the edges, make it difficult to define subtle properties of actions.

By contrast, Dynamic epistemic logic (DEL) [38] was designed to describe actions precisely: how they affect the world and how they are perceived. In particular, classic action types such as public/private announcements or public actions correspond to natural classes of DEL action models. Also, DEL extends epistemic logic and hence enables modelling higher-order knowledge, i.e., what an agent knows about what another agent knows etc, and the evolution of agents’ knowledge over time.

A classification of the complexity with respect to action types was addressed in the literature of epistemic planning, a problem that asks for the existence of a plan, i.e., a finite sequence of DEL actions to reach a situation that satisfies some given objective expressed in epistemic logic. However this problem, which can be seen as solving one-player reachability games with epistemic objective, has never been considered in a strategic, adversarial context. This work bridges the gap between DEL and games by introducing adversarial aspects in DEL planning, thus moving from plan generation to strategy synthesis. We define two frameworks for DEL-based reachability games, where players start in a given epistemic situation and their possible moves are described by action models, and the objective is to reach a situation satisfing some epistemic formula.

In a first step we consider open systems [21], i.e., systems that interact with an environment. In our setting, two omniscient, external entities (that we call controller and environment) choose in turn which actions are performed. We call this setting DEL controller synthesis. Here, agents involved in the models and formulas are not active, they merely observe how the system evolves based on the actions chosen by the controller and the environment, and update their knowledge accordingly. DEL controller synthesis extends DEL planning, as the latter is a degenerate case of the former where the environment stays idle, and we therefore inherit undecidability for the general case. Nevertheless we show that, as for DEL planning, decidability is regained when actions do not increase uncertainty (so-called non-expanding actions) or when the preconditions of actions are propositional formulas. More precisely, we show Pspace -completeness when possible moves are public announcements, Exptime -completeness for the more general public actions, and membership in $(k+1)$-Exptime for propositional actions when the objectives are formulas of modal depth at most $k$.

We then generalise further this setting by turning agents into players. Unlike the omniscient controller of the former setting, agents have imperfect information about the current state of the game, and can only base their decisions on what they know. In the theory of games with imperfect information this is modelled by the notion of uniform strategies, also called observation-based strategies [1]. We study the problem of distributed strategy synthesis, where a group of players cooperate to enforce some objective against the remaining players. As for multi-player games with imperfect information the problem is undecidable, already for propositional actions and a coalition of two players. However we show that the two kinds of assumptions that make imperfect-information games decidable, namely public actions and hierarchical information, also yield decidable cases of multiplayer DEL games. Furthermore, in the case of public announcements and public actions, the complexity is not worse than for controller synthesis.

Related work

The complexity of DEL-based epistemic planning has been thoroughly investigated. It is undecidable already for actions with preconditions of modal depth one and propositional postconditions [9, 22]. For preconditions of modal depth one and no postconditions the problem has been open for years, but it is decidable when pre- and postconditions are propositional [39, 2, 16]. It is also known to be NP -complete for public announcements [10, 14], and Pspace -complete for public actions [14].

The decidability for propositional actions has been extended in [2] by considering infinite trees of actions called protocols instead of finite plans, and specifications in branching-time epistemic temporal logic instead of reachability for epistemic formulas; this has been extended further in [16] by enriching the specification language with Chain Monadic Second-order Logic. Both results rely on the fact that when actions are propositional, the infinite structures generated by repeated application of action models form a class of regular structures [2, 26], i.e., relational structures that have a finite representation via automata. First-order logic is decidable on such structures [8], and chain-MSO is decidable on a subclass called regular automatic trees [16], but neither of these logics can express the existence of strategies in games. However we will show that the regular structures obtained from propositional DEL models can be seen as finite turn-based game arenas studied in games played on graphs. This allows us to transfer decidability results on games with epistemic temporal objectives to the DEL setting.

A notion of cooperative planning in DEL has been studied in [17], but without the adversarial aspect of games. Also, in [23], a game setting has been developed with the so-called Alternating-time Temporal Dynamic Epistemic Logic, but it does not consider uniform strategies and thus cannot express the existence of distributed strategies. Our controller synthesis problem can be expressed in this logic, but not in the fragment that they solve, which cannot express reachability. On the other hand, several decidability results for logics for strategic and epistemic reasoning have been established recently [5, 27], but they do not offer the fine modelling of actions possible in DEL. For instance they cannot easily model public announcements, which we show yield better complexity than those obtained in their settings.

Table 1 sums up previous results for epistemic planning, as well as the results established in this contribution.

2 Background in epistemic planning

Let us fix a countable set of atomic propositions $\mathit{AP}$.

2.1 The classic DEL setting

We recall models of epistemic logic [19].

Definition 2.1**.**

An epistemic model $\mathcal{M}=(W,(R_{{a}})_{{a}\in\textit{Agt}},V)$ is a tuple where

• •

$W$* is a non-empty finite set of possible worlds (or situations),*

• •

$\textit{R}_{{a}}\subseteq W\times W$* is an accessibility relation for agent ${a}$, and*

• •

$V:W\rightarrow 2^{\mathit{AP}}$* is a valuation function.*

We write $w\textit{R}_{{a}}u$ instead of $(w,u)\in\textit{R}_{{a}}$; its intended meaning is that when the actual world is $w$, agent ${a}$ considers that $u$ may be the actual world. The valuation function $V$ provides the subset of atomic propositions that hold in a world. A pair $(\mathcal{M},w)$ is called a pointed epistemic model, and we let $|\mathcal{M}|$ be the size of $\mathcal{M}$, defined as $|W|+\sum_{{a}\in\textit{Agt}}|R_{{a}}|+\sum_{w\in W}|V(w)|$. We will only consider finite models, i.e., we assume that $V(w)$ is finite for all worlds.

The syntax of Epistemic Logic $\mathcal{L}_{\mathbf{EL}}$ is given by the following grammar:

[TABLE]

where $p$ ranges over $\mathit{AP}$ and ${a}$ ranges over Agt.

$K_{{a}}\varphi$ is read ‘agent ${a}$ knows that $\varphi$ is true’. We define the usual abbreviations $(\varphi_{1}\land\varphi_{2})$ for $\lnot(\lnot\varphi_{1}\lor\lnot\varphi_{2})$ and $\hat{K}_{a}\varphi$ for $\lnot K_{a}\lnot\varphi$, and use $\mathcal{L}_{\text{Prop}}$ for the fragment of $\mathcal{L}_{\mathbf{EL}}$ with propositional formulas only. The modal depth of a formula is its maximal number of nested knowledge operators; for instance, the formula $K_{{a}}K_{{b}}p\land\lnot K_{a}q$ has modal depth $2$. The size $|\varphi|$ of a formula $\varphi$ is the number of symbols in it.

The semantics of $\mathcal{L}_{\mathbf{EL}}$ relies on pointed epistemic models.

Definition 2.2**.**

We define $\mathcal{M},w\models\varphi$, read as ‘formula $\varphi$ holds in the pointed epistemic model $(\mathcal{M},w)$’, by induction on $\varphi$, as follows:

• •

$\mathcal{M},w\models p$* if $p\in V(w)$;*

• •

$\mathcal{M},w\models\neg\varphi$* if it is not the case that $\mathcal{M},w\models\varphi$;*

• •

$\mathcal{M},w\models(\varphi\lor\psi)$* if $\mathcal{M},w\models\varphi$ or $\mathcal{M},w\models\psi$;*

• •

$\mathcal{M},w\models K_{{a}}\varphi$* whenever for all $u$ such that $w\textit{R}_{{a}}u$, $\mathcal{M},u\models\varphi$.*

Dynamic Epistemic Logic (DEL) relies on action models (also called “event models”). These models specify how agents perceive the occurrence of an action as well as its effects on the world.

Definition 2.3**.**

An action model $\mathcal{A}=(A,({R^{\mathcal{A}}_{{a}}})_{{a}\in\textit{Agt}},{\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E},\text{post})$ is a tuple where:

• •

$A$* is a non-empty finite set of possible actions,*

• •

${R^{\mathcal{A}}_{{a}}}\subseteq A\times A$* is the accessibility relation for agent ${a}$,*

• •

${\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E}:A\to\mathcal{L}_{\mathbf{EL}}$* provides the precondition for an action to be performed, and*

• •

$\text{post}:A\times\mathit{AP}\to\mathcal{L}_{\text{Prop}}$* provides the postcondition (i.e., the effects) of an action.*

A pointed action model is a pair $(\mathcal{A},\alpha)$ where $\alpha$ represents the actual action. We let $|\mathcal{A}|$ be the size of $\mathcal{A}$, defined as $|\mathcal{A}|:=|A|+\sum_{{a}\in\textit{Agt}}|{R^{\mathcal{A}}_{{a}}}|+\sum_{\alpha\in A}|{\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E}(\alpha)|+\sum_{\alpha\in A,p\in\mathit{AP}}|\text{post}(\alpha,p)|$.

An action $\alpha$ is executable in a world $w$ of an epistemic model $\mathcal{M}$ if $\mathcal{M},w\models{\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E}(\alpha)$, and in that case we define $V(w,\alpha):={\{p\in\mathit{AP}\mid\mathcal{M},w\models\text{post}(\alpha,p)\}}$, the set of atomic propositions that hold after occurrence of action $\alpha$ in world $w$. Since postconditions are always propositional, we can define similarly $V(v,\alpha)$ where $v\subseteq 2^{\mathit{AP}}$ is a valuation.

Types of actions

We identify noticeable types of actions. An action model $\mathcal{A}$ is propositional if all pre- and postconditions of actions in $\mathcal{A}$ belong to $\mathcal{L}_{\text{Prop}}$. A public action is a pointed action model $\mathcal{A},\alpha$ such that for each agent ${a}$, ${R^{\mathcal{A}}_{{a}}}$ is the identity relation. A public announcement is a public action $\mathcal{A},\alpha$ such that for all $p$, $\text{post}(\alpha,p)=p$.

We recall the product that models how to update an epistemic model when an action is executed [4].

Definition 2.4** (Product [4]).**

Let $\mathcal{M}=(W,(R_{{a}})_{{a}\in\textit{Agt}},V)$ be an epistemic model, and $\mathcal{A}=(A,({R^{\mathcal{A}}_{{a}}})_{{a}\in\textit{Agt}},{\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E},\text{post})$ be an action model. The product of $\mathcal{M}$ and $\mathcal{A}$ is defined as $\mathcal{M}\otimes\mathcal{A}=(W^{\prime},(R_{{a}})^{\prime},V^{\prime})$ where:

• •

$W^{\prime}={\{(w,\alpha)\in W\times A\mid\mathcal{M},w\models{\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E}(\alpha)\}}$,

• •

$(w,\alpha)R_{{a}}^{\prime}(w^{\prime},\alpha^{\prime})$* if $wR_{{a}}w^{\prime}$ and $\alpha{R^{\mathcal{A}}_{{a}}}\alpha^{\prime}$, and*

• •

$V^{\prime}((w,\alpha))=V(w,\alpha)$.

Example 2.5**.**

Figure 1 shows the pointed model $\mathcal{M},w$ that represents a situation in which $p$ is true and both agents ${a}$ and ${b}$ do not know it. The pointed action model $\mathcal{A},\alpha$ describes the action where agent ${a}$ learns that $p$ was true but that it is now set to false, while agent ${b}$ does not learn anything (she sees action $\alpha^{\prime}$ that has trivial pre- and postcondition). In the product epistemic model $\mathcal{M}\otimes\mathcal{A},(w,\alpha)$, agent ${a}$ now knows that $p$ is false, while ${b}$ still does not know the truth value of $p$, or whether agent $a$ knows it.

An epistemic model (resp. an action model) is S5 if all accessibility relations are equivalence relations. This property is important to model games with imperfect information, and we will assume it in Section 4.

2.2 Generated structure

Iteratively executing an action model from an initial epistemic model generates an infinite sequence of epistemic models, whose union yields an infinite epistemic structure where dynamics are represented by the possible sequences of actions, while information is captured by the accessibility relations.

Definition 2.6** (Generated structure).**

*Given $\mathcal{M}=(W,\{R_{{a}}\}_{{a}\in\textit{Agt}},V)$ an epistemic model and $\mathcal{A}=(A,\{{R^{\mathcal{A}}_{{a}}}\}_{{a}\in\textit{Agt}},{\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E},\text{post})$ an action model, we define the family of disjoint epistemic models $\{{\mathcal{M}}{\mathcal{A}}^{n}\}_{n\geq 0}$ by letting ${\mathcal{M}}{\mathcal{A}}^{0}=\mathcal{M}$ and ${\mathcal{M}}{\mathcal{A}}^{n+1}={\mathcal{M}}{\mathcal{A}}^{n}\otimes\mathcal{A}$. We finally define the infinite epistemic model ${\mathcal{M}}{\mathcal{A}}^{*}=\bigcup_{n\in\mathbb{N}}{\mathcal{M}}{\mathcal{A}}^{n}$. *

In the following we identify objects of the form $(\ldots((w,\alpha_{1}),\alpha_{2}),\ldots\alpha_{n})$ with $(w,\alpha_{1},\ldots,\alpha_{n})$. Anticipating the game setting we later define, we call a play an infinite sequence $\pi=w\alpha_{1}\alpha_{2}\ldots$ such that all finite prefixes of $\pi$ are in ${\mathcal{M}}{\mathcal{A}}^{*}$. A history is a finite prefix $h$ of a play. We let $\text{Plays}_{{\mathcal{M}}{\mathcal{A}}^{*}}(w)$ and $\text{Hist}_{{\mathcal{M}}{\mathcal{A}}^{*}}(w)$ be, respectively, the set of all plays and histories in ${{\mathcal{M}}{\mathcal{A}}^{*}}$ that start with $w$. These definitions entail the following.

Lemma 2.7**.**

For every world $(w,\alpha_{1},\ldots,\alpha_{n})\in\mathcal{M}\otimes\mathcal{A}^{n}$, and every formula $\varphi\in\mathcal{L}_{\mathbf{EL}}$,

[TABLE]

This shows that the alternative definition of epistemic planning given in the next section is equivalent to the usual one.

2.3 Epistemic planning

The epistemic planning problem asks for the existence of an executable sequence of designated actions $\alpha_{1},\dots,\alpha_{n}$ in an action model $\mathcal{A}$, whose execution from $\mathcal{M},w$ leads to a situation satisfying some objective expressed as an epistemic logic formula. Formally, we consider the following problem.

Definition 2.8** (Plan existence problem).**

• •

Input: a pointed epistemic model $\mathcal{M},w$, an action model $\mathcal{A}$ and an objective formula $\varphi\in\mathcal{L}_{\mathbf{EL}}$;

• •

Output: yes if there is a history $h$ in $\text{Hist}_{{\mathcal{M}}{\mathcal{A}}^{*}}(w)$ such that ${\mathcal{M}}{\mathcal{A}}^{*},h\models\varphi$.

Remark 2.9**.**

Note that usual formulations of the plan existence problem consider a set of distinct pointed action models $(\mathcal{A}_{1},\alpha_{1}),\ldots,(\mathcal{A}_{n},\alpha_{n})$ instead of one action model $\mathcal{A}$. Both formulations are equivalent, in the sense that they are interreducible in linear time.

Main known results on the plan existence problem are summarised in Table 1, while the relevant pointers to the literature are given in the related work paragraph of the introduction.

We recall some standard notions and notations that we will need in the rest of the paper. A finite (resp. infinite) word over some alphabet $\Sigma$ is an element of $\Sigma^{*}$ (resp. $\Sigma^{\omega}$). The length of a finite word $w=w_{0}w_{1}\ldots w_{n}$ is $|w|:=n+1$, and $\mbox{last}(w):=w_{n}$ is its last letter. Given a finite (resp. infinite) word $w$ and $0\leq i<|w|$ (resp. $i\in\mathbb{N}$), we let $w_{i}$ be the letter at position $i$ in $w$, $w_{\leq i}:=w_{0}\ldots w_{i}$ is the prefix of $w$ that ends at position $i$ and $w_{\geq i}:=w_{i}w_{i+1}\ldots$ is the suffix of $w$ that starts at position $i$. We also use variables $x$ that range over some finite domain. We will write $(x=d)$ for the fact “the value of $x$ is $d$”, and use $x:=d$ for the effect of setting $x$ to value $d$. This can all be encoded with atomic propositions.

3 Controller synthesis

We first generalise the plan existence problem to the setting where some environment may perturb the execution of the plan that should thus be robust against it.

Formally, we consider an initial epistemic model $\mathcal{M}$, as in Definition 2.1, with an initial world $w_{\iota}$, and an action model $\mathcal{A}=(A,({R^{\mathcal{A}}_{{a}}})_{{a}\in\textit{Agt}},{\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E},\text{post})$ whose set of actions $A$ is partitioned into actions in $\mathsf{A}_{\text{ctr}}$ controlled by a Controller and actions in $\mathsf{A}_{\text{env}}$ controlled by the Environment.

Controller and Environment play in turn: in each round, Controller first chooses to execute an action in $\mathsf{A}_{\text{ctr}}$, then Environment chooses to execute an action in $\mathsf{A}_{\text{env}}$. Thus instead of seeking a history in ${\mathcal{M}}{\mathcal{A}}^{*}$ that reaches an objective formula, as in epistemic planning, one seeks a strategy for Controller: formally, it is a partial function $\sigma:\text{Hist}_{{\mathcal{M}}{\mathcal{A}}^{*}}(w_{\iota})\rightharpoonup\mathsf{A}_{\text{ctr}}$ defined on histories of odd length (when it is the controller’s turn). An outcome of a strategy $\sigma$ is a play $\pi=w_{\iota}\alpha_{1}\alpha_{2}\ldots$ in which the controller follows $\sigma$, i.e., for all $i\in\mathbb{N}$, $\alpha_{2i+1}=\sigma(\pi_{\leq 2i})\in\mathsf{A}_{\text{ctr}}$, while the other actions, of the form $\alpha_{2i+2}$, are selected by the environment. A strategy $\sigma$ for Controller is winning for an objective formula $\varphi\in\mathcal{L}_{\mathbf{EL}}$ if for every outcome $\pi$ of $\sigma$, there exists $i\in\mathbb{N}$ s.t. ${\mathcal{M}}{\mathcal{A}}^{*},\pi_{\leq i}\models\varphi$.

Definition 3.1** (The controller synthesis problem).**

• •

Input: a pointed epistemic model $\mathcal{M},w_{\iota}$, action model $\mathcal{A}$ with $A=\mathsf{A}_{\text{ctr}}\uplus\mathsf{A}_{\text{env}}$, and an objective $\varphi\in\mathcal{L}_{\mathbf{EL}}$;

• •

Output: yes if there exists a winning strategy for Controller for objective $\varphi$; no otherwise.

Remark 3.2**.**

Formally, we define and study the problem of existence of a strategy. We take the liberty to call the problem controller synthesis because all the algorithms we provide can produce a winning strategy whenever there exists one. The same remark applies to the distributed strategy synthesis problem defined in the next section.

As the plan existence problem reduces to the controller synthesis problem, the undecidability of the former entails the one of the latter. We next establish that in all known subcases where the plan existence problem is decidable, so is the controller synthesis problem.

3.1 The case of non-expanding action models

We consider so-called non-expanding action models where actions do not expand epistemic models when executed, like public actions. For this type of actions, the search space is finite and thus the problem is decidable. We establish the precise computational complexity of the problem in these cases.

Theorem 3.3**.**

When actions are public announcements, the controller synthesis problem is Pspace -complete.

Proof 3.4**.**

Since applying public announcements to epistemic models only removes worlds, and does not change those that remain, the number of successive public announcements to consider can be bounded by the number of worlds in the initial epistemic model. We can thus solve the problem with an alternating algorithm that runs in polynomial time, guessing existentially actions of the controller and universally those of the environment. The algorithm is given in Figure 2.

We conclude by recalling that alternating polynomial time corresponds to deterministic polynomial space [13]. Note that checking epistemic formulas (preconditions and $\varphi$) in epistemic models, and thus also computing the update product, can be performed in polynomial time.

PSPACE-hardness comes from a polynomial reduction from TQBF (True Quantified Boolean Formulae) which is PSPACE-complete [35]. A QBF formula

[TABLE]

is transformed in the following instance of the controller existence problem:

• •

$\mathcal{M}$* is the pointed Kripke model made up of a ${\{q_{i}\}}$-world and ${\{p_{i}\}}$-world for all $i\in{\{1,\dots,2k\}}$ and an extra $\emptyset$-world $w$, which is the pointed world; the epistemic relation for agent ${a}$ is universal;*

w:\emptyset$$\vdots$$\vdots$$w_{1}:{\{p_{1}\}}$$u_{1}:{\{q_{1}\}}$$w_{2}:{\{p_{2}\}}$$u_{2}:{\{q_{2}\}}$$w_{2k}:{\{p_{2k}\}}$$u_{2k}:{\{q_{2k}\}}

• •

The possible announcements are

[TABLE]

for $i\in{\{1,\dots,2k\}}$.

They belong to the controller when $i$ is odd, and to the environment when $i$ is even;

• •

The goal is $\bigwedge_{j=1}^{2k}K_{{a}}\lnot q_{j}\land\chi(\hat{K}_{{a}}p_{1},\dots,\hat{K}_{{a}}p_{2k})$.

In the model $\mathcal{M}$, worlds $w_{i}$ are used to encode assignments of truth values to atoms $p_{i}$: removing world $w_{i}$ means setting $p_{i}$ to true, while keeping it means setting $p_{i}$ to false. Worlds $u_{i}$, bearing atoms $q_{i}$, are used to enforce that the value of each atom $p_{i}$ is set exactly once. In announcements $\varphi_{p_{i}}$ and $\varphi_{\neg p_{i}}$, conjunct $\bigwedge_{j=1}^{i-1}K_{{a}}\lnot q_{j}\land\bigwedge_{j=i}^{2k}\hat{K}_{{a}}q_{j}$ implies that worlds $u_{1},\ldots,u_{i-1}$ have already been removed, while worlds $u_{i},\ldots,u_{2k}$ are still in the model. Thus announcements $\varphi_{p_{i}}$ and $\varphi_{\neg p_{i}}$ are possible in round $i$, and only there.

Now observe that announcement $\varphi_{\neg p_{i}}$, because of conjunct $\neg p_{i}\land\neg q_{i}$, removes both world $w_{i}$ and world $u_{i}$, thus setting $p_{i}$ to true. Announcement $\varphi_{p_{i}}$ instead removes only world $u_{i}$, thus setting $p_{i}$ to true.

In the goal formula, $\bigwedge_{j=1}^{2k}K_{{a}}\lnot q_{j}$ means that all the variables $p_{1},\dots,p_{2k}$ have been assigned. The clause $\chi(\hat{K}_{{a}}p_{1},\dots,\hat{K}_{{a}}p_{2k})$ is the formula $\chi(p_{1},\dots,p_{2k})$ in which we replaced $p_{i}$ by $\hat{K}_{{a}}p_{i}$, which holds if and only if world $w_{i}$ has not been removed by announcements, i.e., if and only if announcement $\varphi_{p_{i}}$ was chosen at round $i$.

The fact that the announcements that assign values to $p_{1},p_{3},\dots$ are assigned to the controller and that the announcements that assign values to $p_{2},p_{4},\dots$ are played by the environment reflects the alternation of quantifiers in the formula $\exists p_{1}\forall p_{2}\dots\exists p_{2k-1}\forall p_{2k}\chi(p_{1},\dots,p_{2k})$.

Theorem 3.5**.**

The controller synthesis problem for public actions is Exptime -complete.

Proof 3.6**.**

As for public announcements, applying a public action in a model does not add worlds. However it may change facts in worlds, so that sequences of actions of linear length may not suffice. Nonetheless, linear space is enough to store the current pointed epistemic model, and we can turn the alternating algorithm from the proof of Theorem 3.3 into one that runs in polynomial space. The new algorithm is given in Figure 3, in which we do not bound the length of the sequence of actions. Note that the algorithm may not terminate, but it is folklore that we can add a counter to ensure termination while staying in polynomial-space; we do not present these tedious technicalities here.

The Exptime -membership of the problem follows from the fact that alternating polynomial space corresponds to exponential time [13].

Exptime* -hardness is obtained by reduction from the conditional planning problem, a variant of classical planning with full observability and non-deterministic actions, where the plan should lead to a situation satisfying the goal no matter how nondeterminism is resolved. However the plan can depend on how nondeterminism is resolved, hence the name “conditional plan” [24, 34].*

Stated in our terms, conditional planning essentially corresponds to a particular case of controller synthesis which is purely boolean (no epistemic content), but where actions chosen by the controller have nondeterministic effects, and the environment resolves nondeterminism. Since everything is purely boolean, the initial situation is a one-world epistemic model, i.e., a valuation over a finite set of atoms $\mathit{AP}$, the goal is a boolean formula over $\mathit{AP}$, and each action is a one-state action model with nondeterministic postcondition. A conditional plan is then a winning strategy for the controller. Thus, to finish the reduction, we only have to show how to simulate nondeterministic actions in our setting.

In [24, 34], a nondeterministic action is modelled as a tuple $\langle\varphi,\overrightarrow{\text{post}}\rangle$ where $\varphi$ is a Boolean precondition, and $\overrightarrow{\text{post}}$ is a finite set ${\{\text{post}_{1},\ldots,\text{post}_{n}\}}$, where $\text{post}_{i}:\mathit{AP}\to\mathcal{L}_{\text{Prop}}$ is a postcondition. The idea is that in each round the controller chooses an action among those whose precondition is true, and the environment resolves the non-determinism by choosing which postcondition of $\overrightarrow{\text{post}}$ to apply to the current valuation. For each nondeterministic action $\langle\varphi,{\{\text{post}_{1},\ldots,\text{post}_{n}\}}\rangle$ of the conditional planning instance, we create one action model for the controller that stores in a finite-domain variable $action$ which action has been played, and $n$ actions for the environment that correspond to the different possible poscondtions. The action for the controller is defined as follows:

[TABLE]

while the actions for the environment are, for each $i\in{\{1,\ldots,n\}}$,

[TABLE]

This finishes the proof, and also shows how the controller synthesis problem subsumes conditional planning. We now present an alternative proof that reduces from a more basic decision problem called $G_{4}$, introduced by Chandra and Stockmeyer [36]. This is essentially an adaptation of the proof from [24] for the EXPTIME-hardness of conditional planning.

The input to the $G_{4}$ problem is a 13-DNF formula over $2k$ atomic propositions $p_{1},\dots,p_{k}$, $q_{1},\dots,q_{k}$ and an initial valuation. Atoms $p_{1},\dots,p_{k}$ are controlled by the controller (the existential player) while $q_{1},\dots,q_{k}$ are controlled by the environment (the universal player). Now, the following game is played: each player, when it is her turn to play, flips the assignment of one of the variables she controls, and turns alternate. The game stops when the 13-DNF formula becomes true, and the winner is the player that made the last move. An instance of the $G_{4}$ problem is positive if the controller has a winning strategy.

We construct the following instance of our controller synthesis problem. The initial epistemic model is made up of one world, whose valuation is the initial valuation of $G_{4}$. Actions of the controller are:

\scriptsize\begin{array}[]{l}pre:\top\\ post:p_{1}:=\lnot p_{1}\end{array}\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\ \dots\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\ \scriptsize\begin{array}[]{l}pre:\top\\ post:p_{k}:=\lnot p_{k}\end{array}**

Actions of the environment are:

\scriptsize\begin{array}[]{l}pre:\top\\ post:q_{1}:=\lnot q_{1}\end{array}\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\ \dots\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\ \scriptsize\begin{array}[]{l}pre:\top\\ post:q_{k}:=\lnot q_{k}\end{array}**

The goal is the 13-DNF formula.

Theorem 3.5 also generalises to other non-expanding actions models such as the so-called separable action models [14], where the preconditions of any two actions in the same connected component are logically inconsistent.

3.2 The case of propositional action models

To solve our controller synthesis problem we rely on the approach followed in [26] to solve the plan existence problem for propositional actions. This approach has two main ingredients: (I1) when $\mathcal{A}$ is propositional, the generated structure ${\mathcal{M}}{\mathcal{A}}^{*}$ can be represented finitely, and (I2) one can decide the existence of a winning strategy in a certain class of two-player games with epistemic objectives.

Theorem 3.7**.**

When action models are propositional, the controller synthesis problem is decidable, and in $(k+1)$-Exptime if the objective’s modal depth is bounded by $k$.

We devote the rest of this section to prove Theorem 3.7, which requires to introduce particular game arenas.

Definition 3.8**.**

A two-player epistemic game arena is a structure $\mathcal{G}=(W,w_{\iota},\Delta,(R_{{a}})_{{a}\in\textit{Agt}},V)$ where $(W,(R_{{a}})_{{a}\in\textit{Agt}},V)$ is an epistemic model, $W=W_{0}\uplus W_{1}$ is partitioned into the positions of players 0 and 1, $w_{\iota}$ is an initial world and $\Delta\subseteq W\times W$ is a transition relation.

A play in a game arena $\mathcal{G}$ is an infinite sequence of worlds $\pi=w_{0}w_{1}w_{2}\ldots$ such that for all $i\in\mathbb{N}$, $w_{i}\Delta w_{i+1}$, and a history is a finite nonempty prefix of a play. We let $\text{Plays}_{\mathcal{G}}$ and $\text{Hist}_{\mathcal{G}}$ be the sets of plays and histories in $\mathcal{G}$, respectively. Accessibility relations $(R_{{a}})_{{a}\in\textit{Agt}}$ are extended to histories to interpret epistemic formulas: two histories $h=w_{0}\ldots w_{n}$ and $h^{\prime}=w^{\prime}_{0}\ldots w^{\prime}_{m}$ are related by $R_{{a}}$ whenever $n=m$ (same length) and $w_{i}R_{{a}}w^{\prime}_{i}$ for every $i\leq n$.

A strategy for player [math] is a partial function $\sigma:\text{Hist}_{\mathcal{G}}\rightharpoonup W$ such that for every $h$ with $\mbox{last}(h)\in W_{0}$: $\mbox{last}(h)\Delta\sigma(h)$. A play $\pi=w_{\iota}w_{1}w_{2}\ldots$ is an outcome of $\sigma$ if for every $i\in\mathbb{N}$ with $\pi_{i}\in W_{1}$, we have $\pi_{i+1}=\sigma(\pi_{\leq i})$. Strategy $\sigma$ is winning for an epistemic objective $\varphi\in\mathcal{L}_{\mathbf{EL}}$, if for every outcome $\pi$ of $\sigma$ there is some $i\in\mathbb{N}$ with $\pi_{\leq i}\models\varphi$.

Theorem 3.9** ([12]).**

The existence of a winning strategy for player 0 in an epistemic game $\mathcal{G}$ for an epistemic objective $\varphi$ of modal depth $k$ can be decided in time $k$-exponential in $|\mathcal{G}|$ and $|\varphi|$.

We show that the controller synthesis problem for propositional action models reduces to solving an epistemic game:

Proposition 3.10**.**

*Given an instance $((\mathcal{M},w_{\iota}),\mathcal{A},\varphi)$ of the controller synthesis problem where $\mathcal{A}$ is propositional, one can construct a game arena $\mathcal{G}$ such that Controller wins in $((\mathcal{M},w_{\iota}),\mathcal{A},\varphi)$ iff Player 0 wins in $\mathcal{G}$ for objective $\varphi$ and $|\mathcal{G}|\leq|\mathcal{M}|+|\mathcal{A}|\times 2^{m+1}$, where $m$ is the number of atomic propositions involved in $\mathcal{M},\mathcal{A}$ and $\varphi$. *

Proof 3.11**.**

Let $\mathcal{M}=(W,(R_{{a}})_{{a}\in\textit{Agt}},V)$ and $\mathcal{A}=(A,\{{R^{\mathcal{A}}_{{a}}}\}_{{a}\in\textit{Agt}},{\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E},\text{post})$, and let $\mathit{AP}_{u}$ be the atomic propositions involved. The worlds of the game arena $\mathcal{G}$ that we build are either worlds $w\in\mathcal{M}$ or tuples $(\alpha,v,i)$ where $\alpha\in\mathcal{A}$ represents the last action performed, $v\in 2^{\mathit{AP}_{u}}$ is the current valutation, and $i\in\{0,1\}$ indicates whose turn it is to play: 0 for Controller and 1 for Environment. In an initial world $w$, Controller can choose an action $\alpha\in\mathsf{A}_{\text{ctr}}$ such that $w\models{\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E}(\alpha)$ and move to $(\alpha,V(w,\alpha),1)$; in a world of the form $(\alpha,v,i)$, if $i=1$ (resp., $i=0$), Environment (resp, Controller) chooses an action $\alpha^{\prime}\in\mathsf{A}_{\text{env}}$ (resp., $\alpha^{\prime}\in\mathsf{A}_{\text{ctr}}$) such that $v\models{\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E}(\alpha^{\prime})$, and moves to $(\alpha^{\prime},V(w,\alpha),1-i)$.

Formally, letting $A_{0}=\mathsf{A}_{\text{ctr}}$ and $A_{1}=\mathsf{A}_{\text{env}}$, we define the epistemic game arena $\mathcal{G}=(W^{\prime},w_{\iota},\Delta,(R_{{a}}^{\prime})_{{a}\in\textit{Agt}},V^{\prime})$ as follows: Let $W_{0}=W\cup A\times 2^{\mathit{AP}}\times\{0\}$, $W_{1}=A\times 2^{\mathit{AP}}\times\{1\}$, and $W^{\prime}=W_{0}\cup W_{1}$. For transitions, for $w\in W$, $\alpha,\alpha^{\prime}\in A$ and $v,v^{\prime}\in 2^{\mathit{AP}}$, we let $w\Delta(\alpha,v,1)$ if $\alpha\in\mathsf{A}_{\text{ctr}},w\models{\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E}(\alpha)$ and $v=V(w,\alpha)$, and $(\alpha,v,i)\Delta(\alpha^{\prime},v^{\prime},1-i)$ if $\alpha^{\prime}\in A_{i},v\models{\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E}(\alpha^{\prime})$ and $v^{\prime}=V(v,\alpha)$. We also let $wR_{{a}}^{\prime}w^{\prime}$ if $wR_{{a}}^{\prime}w^{\prime}$, and $(\alpha,v,i)R_{{}^{\prime}}(\alpha^{\prime},v^{\prime},i)$ if $\alpha{R^{\mathcal{A}}_{{a}}}\alpha^{\prime}$, and finally $V^{\prime}(w)=V(w)$ and $V^{\prime}(\alpha,v,i)=v$.

The structure given by the set of histories $\text{Hist}_{\mathcal{G}}$ and relations $R_{{a}}^{\prime}$ extended to these histories is isomorphic to ${\mathcal{M}}{\mathcal{A}}^{*}$, and histories of odd (resp. even) length in ${\mathcal{M}}{\mathcal{A}}^{*}$ correspond to histories that end in $W_{0}$ (resp. $W_{1}$) in $\mathcal{G}$ (provided they start in $w_{\iota}$). It follows that there is a winning strategy for Controller in the original problem if and only if there is winning strategy for Player 0 in $\mathcal{G}$ with objective $\varphi$.

Note that, as stated in Proposition 3.10, the resulting game arena is indeed polynomial in the size of the epistemic and action models, but it is exponential in the number of atomic propositions involved in the problem. This is because states of the game arena contain all possible valuations. Theorem 3.7 now follows from Theorem 3.9 and Proposition 3.10.

With the controller synthesis problem we enriched epistemic planning with an adversarial environment. Still, as in epistemic planning, the agents are mere observers. We now make a step further and make the agents players of the game.

4 Distributed strategy synthesis

In this section agents are no more passive, but instead they are players who choose themselves the actions that occur. The set Agt of agents is split into two teams $\textit{Agt}_{\exists}$ and $\textit{Agt}_{\forall}$ that play against each other, and we may say players instead of agents.

4.1 Setting up the game

Unlike the external controller from the previous section, our players now have imperfect information. The fundamental feature of games with imperfect information is that when a player cannot distinguish between two different situations, a strategy for this player should prescribe the same action in both situations. All the additional complexity in solving games with imperfect information compared to the perfect information setting arises from this constraint. Such strategies are often called uniform or observation-based (see for instance [33, 37, 1]). Since games with imperfect information consider S5 models, i.e., where accessibility relations are equivalence relations, and it is unclear what uniform strategies mean in non-S5 models111Actually the usual definition seems to make sense for KD45, i.e. models whose relations are serial, transitive, and Euclidean. But this would be highly non-standard, and since all the literature on games with imperfect information considers S5 models., we also assume from now on that all epistemic and action models are S5. We stress this assumption by writing $\sim_{{a}}$ (resp. $\sim^{\mathcal{A}}_{{a}}$) instead of $R_{{a}}$ (resp. ${R^{\mathcal{A}}_{{a}}}$).

We start from an initial pointed epistemic model $\mathcal{M},w$, and an action model $\mathcal{A}$ whose set of actions is partitioned into subsets $(A_{{a}})_{{a}\in\textit{Agt}}$ of actions for each player. The game we describe is turn-based. We use the variable $\mathsf{turn}$ ranging over Agt to represent whose turn it is to play. We require that for each ${a}\in A_{{a}}$, ${\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E}(\alpha)$ implies $(\mathsf{turn}{=}{a})$, and that postconditions for variable $\mathsf{turn}$ do not depend on the current world, but instead the next value of $\mathsf{turn}$ is completely determined by the action only.

Moreover, in order to obtain a proper imperfect-information game, we demand the following hypotheses:

Hypotheses on $\mathcal{M}$ and $\mathcal{A}$

1. (H1)

The starting player is known: there is a player ${a}$ such that for all $u\in W$, $\mathcal{M},u\models(\mathsf{turn}{=}{a})$; 2. (H2)

The turn stays known: for all actions $\alpha,\alpha^{\prime}$ and agent ${a}$, if $\alpha R_{{a}}\alpha^{\prime}$, then $\alpha$ and $\alpha^{\prime}$ assign the same value to $\mathsf{turn}$. 3. (H3)

Players know their available actions: if agent ${a}$ can execute $\alpha$ after history $h$, then she can also execute it after every history $h^{\prime}$ with $h\sim_{{a}}h^{\prime}$.

All these hypotheses can be either enforced syntactically or checked in the different decidable cases we consider in the rest of this work (see the long version for detail).

We now define formally the notion of uniform strategies.

Definition 4.1** (Uniform strategy).**

A strategy $\sigma$ for player ${a}$ is uniform if for every pair of histories $h,h^{\prime}$ where it is player ${a}$’s turn, $h\sim_{{a}}h^{\prime}$ entails $\sigma(h)=\sigma(h^{\prime})$.

In the rest of this section, a strategy of a player in $\textit{Agt}_{\exists}$ is implicitly uniform. When one selects a strategy for each player in $\textit{Agt}_{\exists}$, the result is called a distributed strategy, and an outcome of a distributed strategy is a play in which all players in $\textit{Agt}_{\exists}$ follow their prescribed strategy. A distributed strategy is winning for an objective formula $\varphi$ if all its outcomes eventually satisfy $\varphi$.

4.2 The distributed strategy synthesis problem

We study the existence of a distributed strategy for players in $\textit{Agt}_{\exists}$ that ensures to reach an epistemic goal property.

Definition 4.2** (Distributed strategy existence problem).**

• •

Input: a pointed epistemic model $\mathcal{M},w$ and an action model $\mathcal{A}$ partitioned into $(A_{{a}})_{{a}\in\textit{Agt}}$ that satisfy hypotheses (H1)-(H3), and an objective formula $\varphi\in\mathcal{L}_{\mathbf{EL}}$;

• •

Output: yes if there exists a winning distributed strategy for players in $\textit{Agt}_{\exists}$; no otherwise.

Unlike the controller synthesis problem which we proved decidable for propositional actions, synthesising distributed strategies is undecidable for propositional actions, already for a team of two players.

4.3 Undecidability for two existential players

The following Theorem 4.3 is a reformulation in our setting of the classical undecidability result from Reif and Peterson [29]. However, we decide to promote an existing elegant reformulation of that very same result, called TEAM DFA GAME [15, Def. 1, p. 14:7], that can be reduced to our distributed strategy synthesis problem.

Theorem 4.3**.**

*The distributed strategy synthesis problem is undecidable, already for a propositional action model and two existential players against one universal player. *

Proof 4.4**.**

The proof is given by reduction from the problem TEAM DFA GAME [15, Def. 1, p. 14:7], shown to be undecidable.

We consider a two-versus-one (players $a$ and $b$ versus player $\forall$) team game played on a deterministic finite automaton (DFA) ${\mathcal{A}}$ whose alphabet is ${\{0,1\}}$, whose set of states is $Q$, initial state is $q_{0}$, transition function $\delta$. Special subsets of states $F_{\exists}$ and $F_{\forall}$ are given. The game starts with ${\mathcal{A}}$ being in state $q_{0}$. Each round is divided in six steps:

if the current state $q$ is in $F_{\exists}$ then team ${\{a,b\}}$ wins; if the current state $q$ is in $F_{\forall}$ then team ${\{\forall\}}$ wins; 2. 2.

Player $\forall$ inputs two bits $\beta,\beta^{\prime}$ into ${\mathcal{A}}$; 3. 3.

Player $a$ learns $\beta$; 4. 4.

Player $a$ inputs one bit $m$ into ${\mathcal{A}}$; 5. 5.

Player $b$ learns $\beta^{\prime}$; 6. 6.

Player $b$ inputs one bit $m^{\prime}$ into ${\mathcal{A}}$.

At each step, player $\forall$ has perfect information. TEAM DFA GAME is the decision problem: given an DFA ${\mathcal{A}}$, subsets of states $F_{\exists}$, $F_{\forall}$, does the team $\{a,b\}$ have a winning strategy?

The rest of the proof consists in representing the initial situation, the game rules and the goal of a TEAM DFA GAME instance as a distributed strategy existence problem instance.

**Definition of the reduction. Let $({\mathcal{A}},F_{\exists},F_{\forall})$ be an instance of TEAM DFA GAME. Teams are $\textit{Agt}_{\exists}={\{a,b\}}$ and $\textit{Agt}_{\forall}={\{\forall\}}$. We introduce a finite-domain variable $q$ that ranges over the set of states of ${\mathcal{A}}$. The variable $q$ can be represented by a finite set of atomic propositions: for example, for an automaton with 8 states from **{0,…,7}, three atomic propositions, $bit_{1}(q)$, $bit_{2}(q)$ and $bit_{3}(q)$ so that say $(q=5)$ is the Boolean formula $bit_{1}(q)\land\neg bit_{2}(q)\land bit_{3}(q)$. We prefer to keep with a finite-domain variable $q$ to avoid technicalities.

We also introduce a finite-domain variable $\mathsf{stp}$ that ranges over ${\{1,2,3,4,5,6\}}$. The Boolean variable $\mathsf{lost}$ is true if the team $\textit{Agt}_{\exists}$ has lost. We define $\mathcal{M},w$ to be the single-world S5 epistemic model in which $(\mathsf{turn}{=}\forall)$, $(q{=}q_{0})$, $(\mathsf{stp}{=}1)$, $\lnot\mathsf{lost}$. The actions in $A_{\forall}$ form an $a$- and $b$-indistinguishably equivalence class and are of the form:

• •

Phys. Rev. E*: $(\mathsf{turn}{=}\forall)\land(\mathsf{stp}{=}1)\land q{\in}F_{\forall}$; post: $\mathsf{lost}{:=}\top,\mathsf{stp}{:=}2$*

• •

Phys. Rev. E*: $(\mathsf{turn}{=}\forall)\land(\mathsf{stp}{=}1)\land q{\not\in}F_{\forall}$; post: $\mathsf{stp}{:=}2$*

• •

Phys. Rev. E*: $(\mathsf{turn}{=}\forall){\land}(\mathsf{stp}{=}2)$; post: $\beta{:=}0,\beta^{\prime}{:=}0,q{:=}\delta(q,00),\mathsf{turn}{:=}a,\mathsf{stp}{:=}{3}$*

• •

Phys. Rev. E*: $(\mathsf{turn}{=}\forall){\land}(\mathsf{stp}{=}2)$; post: $\beta{:=}1,\beta^{\prime}{:=}0,q{:=}\delta(q,10),\mathsf{turn}{:=}a,\mathsf{stp}{:=}{3}$*

• •

Phys. Rev. E*: $(\mathsf{turn}{=}\forall){\land}(\mathsf{stp}{=}2)$; post: $\beta{:=}0,\beta^{\prime}{:=}1,q{:=}\delta(q,01),\mathsf{turn}{:=}a,\mathsf{stp}{:=}{3}$*

• •

Phys. Rev. E*: $(\mathsf{turn}{=}\forall){\land}(\mathsf{stp}{=}2)$; post: $\beta{:=}1,\beta^{\prime}{:=}1,q{:=}\delta(q,11),\mathsf{turn}{:=}a,\mathsf{stp}{:=}{3}$*

$A_{a}$* is a $b$-indistinguishably equivalence class and contains:*

• •

Phys. Rev. E*: $(\mathsf{turn}{=}a)\land(\mathsf{stp}{=}3)\land\beta$; post: $\mathsf{stp}:=4$*

• •

Phys. Rev. E*: $(\mathsf{turn}{=}a)\land(\mathsf{stp}{=}3)\land\lnot\beta$; post: $\mathsf{stp}:=4$*

• •

Phys. Rev. E*: $(\mathsf{turn}{=}a)\land(\mathsf{stp}{=}4)$; post: $m{:=}\bot;\mathsf{stp}:=5;q:=\delta(q,0);\mathsf{turn}:=b$*

• •

Phys. Rev. E*: $(\mathsf{turn}{=}a)\land(\mathsf{stp}{=}4)$; post: $m{:=}\top;\mathsf{stp}{:=}5;q{:=}\delta(q,1);\mathsf{turn}:=b$*

$A_{b}$* is an $a$-indistinguishably equivalence class and contains:*

• •

Phys. Rev. E*: $(\mathsf{turn}{=}b)\land(\mathsf{stp}{=}5)\land\beta^{\prime}$; post: $\mathsf{stp}{:=}6$*

• •

Phys. Rev. E*: $(\mathsf{turn}{=}b)\land(\mathsf{stp}{=}5)\land\lnot\beta^{\prime}$; post: $\mathsf{stp}{:=}6$*

• •

Phys. Rev. E*: $(\mathsf{turn}{=}b)\land(\mathsf{stp}{=}6)$; post: $m^{\prime}{:=}\bot;\mathsf{stp}{:=}1;q{:=}\delta(q,0);\mathsf{turn}{:=}\forall$*

• •

Phys. Rev. E*: $(\mathsf{turn}{=}b)\land(\mathsf{stp}{=}4)$; post: $m^{\prime}{:=}\top;\mathsf{stp}{:=}1;q{:=}\delta(q,1);\mathsf{turn}{:=}\forall$*

The assignments $q:=\delta(q,0)$ and $q:=\delta(q,1)$ are shortcuts for some “if statements” on states, e.g. ‘if $q=5$, then $q:=2$’. For instance, assuming that we have eight states $\{s_{0},\ldots,s_{7}\}$ which are thus representable with three bits, the assignment $q:=\delta(q,0)$ is simulated by the following set of propositional assignments: ${\{bit_{i}(q):=\psi_{i}\}}_{i=1..3}$, where $\psi_{i}$ is the Boolean formula \bigvee_{k\in{0..7}\text{ s.t. the i-th bit of \delta(s_{k},0) is 1}}(q=s_{k}).

The goal formula $\varphi$ is $\lnot\mathsf{lost}\land(\mathsf{stp}{=}1)\land(q{\in}F_{\exists})$.

We now turn to decidable cases: games with imperfect information and epistemic objectives are known to be decidable either when actions are public [6], or when information is hierarchical [27]. We establish similar results in our setting.

4.4 The case of non-expanding action models

Theorems 3.3 and 3.5 of Section 3 generalise to the distributed strategy synthesis problem. First, we inherit the lower bounds by letting $\textit{Agt}_{\exists}=\{\text{Controller}\}$ and $\textit{Agt}_{\forall}=\{\text{Environment}\}$, and by making them alternate turns. Second, the upper bounds are obtained by adapting the alternating algorithms for the upper bounds of Theorems  3.3 and 3.5. We need to ensure that existential choices of actions of an agent ${a}\in\textit{Agt}_{\exists}$ lead to a uniform strategy. To do that, every time agent ${a}$ picks an action $\alpha$, we perform an extra universal choice over $\sim_{{a}}$-indistinguishable worlds, and continue executing the algorithm from these worlds.

Theorem 4.5**.**

For public announcements, the distributed strategy synthesis problem is PSPACE-complete.

Proof 4.6**.**

The alternating algorithm that runs in polynomial time is similar to the one given in the proof of Theorem 3.3, except that we also add universal choices of ${R^{\mathcal{A}}_{{a}}}$-successors for player ${a}$ in $\textit{Agt}_{\exists}$. More precisely, the steps of the algorithm works as follows. When it is the turn of a player ${a}$ in $\textit{Agt}_{\forall}$ to play, simply universally guess an executable action in $A_{{a}}$. When it is the turn of a player ${a}$ in $\textit{Agt}_{\exists}$ to play, then perform the following steps:

• •

first existentially guess an executable action $\alpha$ in $A_{{a}}$;

• •

second, universally guess a $R_{{a}}$-successor of the pointed world in the current epistemic model and make it as the new pointed world;

• •

compute the new epistemic model by executing the $\alpha$.

As for Theorem 3.3, the length of such a sequence is bounded by the number of worlds in the initial epistemic model $\mathcal{M}$. The obtained algorithm is given in Figure 4.

Hardness follows from Theorem 3.3 (simply consider the controller as the single agent in $\textit{Agt}_{\exists}$, the environment as the single agent in $\textit{Agt}_{\forall}$, and make them alternate).

Theorem 4.7**.**

For public actions, the distributed strategy synthesis problem is EXPTIME-complete.

Proof 4.8**.**

The alternating algorithm that runs in polynomial space is similar to the one given in the proof of Theorem 3.3. The algorithm is in polynomial space for the same reason said in the proof of Theorem 3.5. The obtained algorithm is given in Figure 5.

Hardness follows from Theorem  3.5.

We now turn to a decidable case for propositional actions.

4.5 Propositional actions+hierarchical information

We consider propositional action models, which may make the size of epistemic models grow unboundedly, but where the information of the different players is hierarchical, making it easier to synchronise the existential players’ strategies.

According to Theorem 4.3, the distributed strategy synthesis problem is undecidable for propositional actions and a two-player team $\textit{Agt}_{\exists}=\{a,b\}$ against team $\textit{Agt}_{\forall}=\{\forall\}$. Observe that in the proof of Theorem 4.3, the information of players $a$ and $b$ is incomparable: in each round $a$ only learns the first bit produced by $\forall$’s move, while $b$ only learns the second bit. This cannot be simulated in games with so-called hierarchical information, a classic restriction to regain decidability in multi-player games of imperfect information [28, 30].

We say that an input of the distributed strategy synthesis problem $((\mathcal{M},w_{\iota}),\mathcal{A},\varphi)$ presents hierarchical information if the set of $\textit{Agt}_{\exists}$ can be totally ordered (${a}_{1}<\ldots<{a}_{n}$) so that $\sim_{{a}_{i}}\leavevmode\nobreak\ \!\!\!\subseteq\leavevmode\nobreak\ \!\!\!\sim_{{a}_{i+1}}$ and $\sim^{\mathcal{A}}_{{a}_{i}}\leavevmode\nobreak\ \!\!\!\subseteq\leavevmode\nobreak\ \!\!\!\sim^{\mathcal{A}}_{{a}_{i+1}}$, for each $1\leq i<n$.

Theorem 4.9**.**

Distributed strategy synthesis with propositional actions and hierarchical information is decidable.

We end the section by sketching the proof of Theorem 4.9. We start by introducing a multi-player variant of the epistemic game arenas from Definition 3.8.

Definition 4.10**.**

A multi-player epistemic game arena $\mathcal{G}=(W,w_{\iota},\Delta,(\sim_{{a}})_{{a}\in\textit{Agt}},V)$ is a structure such that

• •

$(W,(\sim_{{a}})_{{a}\in\textit{Agt}},V)$* is an epistemic model,*

• •

$W=\uplus_{{a}\in\textit{Agt}}W_{a}$* is partitioned into positions of each agent,*

• •

$w_{\iota}$* is an initial world and*

• •

$\Delta\subseteq W\times W$* is a transition relation.*

Accessibility relations $\sim_{{a}}$ are extended to histories, strategies of agent $a$ are required to be uniform with respect to $\sim_{{a}}$, and the notions of outcomes, distributed strategies and winning distributed strategies are defined as before.

Theorem 4.9 is established by reducing the distributed strategy synthesis problem to a similar problem in multi-player epistemic games, known to be decidable:

Theorem 4.11** ([31, 27]).**

*Existence of winning distributed strategies in multi-player epistemic games with hierarchical information and epistemic temporal objectives is decidable. *

The reduction is very similar to the one in the proof of Proposition 3.10. The main difference is that we use variable $\mathsf{turn}$ instead of bit $i\in\{0,1\}$ to define the positions of the different agents. The imperfect information of players is defined based on the accessibility relations in $\mathcal{M}$ and $\mathcal{A}$. More precisely, we can show that:

Proposition 4.12**.**

Given an instance $((\mathcal{M},w_{\iota}),\mathcal{A},\varphi)$ of the distributed strategy synthesis problem where $\mathcal{A}$ is propositional, one can construct a multi-player epistemic game arena $\mathcal{G}$ such that the distributed strategy synthesis problem is equivalent to the existence of a winning distributed strategy for $\textit{Agt}_{\exists}$ to enforce $\varphi$ in $\mathcal{G}$ and $|\mathcal{G}|\leq|\mathcal{M}|+|\mathcal{A}|\times 2^{m}$, where $m$ is the number of atomic propositions involved.

Proof 4.13**.**

Let $\mathcal{M}=(W,(\sim_{{a}})_{{a}\in\textit{Agt}},V)$ be an S5 epistemic model, $\mathcal{A}=(A,\{\sim^{\mathcal{A}}_{{a}}\}_{{a}\in\textit{Agt}},{\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E},\text{post})$ an S5 propositional event model with $A=\uplus_{{a}\in\textit{Agt}}A_{a}$, and $w_{\iota}\in W$ an initial world. Let $\mathit{AP}_{u}$ be the atomic propositions involved.

We define the multiplayer epistemic game arena $\mathcal{G}=(W^{\prime},w_{\iota},\Delta,(\sim_{{a}}^{\prime})_{{a}\in\textit{Agt}},V^{\prime})$ as follows. First, let $W^{\prime}=W\cup A\times 2^{\mathit{AP}}$, and for each ${a}\in\textit{Agt}$ let $W^{\prime}_{a}={\{w\mid w\models(\mathsf{turn}{=}{a})\}}\cup{\{(\alpha,v)\mid v\models(\mathsf{turn}{=}{a})\}}$. Next, for transitions, we let $(w,(\alpha,v))\in\Delta$ if $w\models{\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E}(\alpha)$ and $v=V(w,\alpha)$, and $((\alpha,v),(\alpha^{\prime},v^{\prime}))\in\Delta$ if $v\models{\rm Phys.\leavevmode\nobreak\ Rev.\leavevmode\nobreak\ E}(\alpha^{\prime})$ and $v^{\prime}=V(v,\alpha)$. Next, for each ${a}\in\textit{Agt}$, we let $\sim_{{a}}^{\prime}\;=\;\sim_{{a}}{}\cup\{((\alpha,v),(\alpha^{\prime},v^{\prime}))\mid(\alpha,\alpha^{\prime})\in{R^{\mathcal{A}}_{{a}}}\}$, and finally $V^{\prime}(w)=V(w)$ and $V^{\prime}(\alpha,v)=v$.

One can see that the structure given by the set of histories $\text{Hist}_{\mathcal{G}}$ and relations $\sim_{{a}}^{\prime}$ extended to these histories is isomorphic to ${\mathcal{M}}{\mathcal{A}}^{*}$, and histories that satisfy $(\mathsf{turn}{=}{a})$ in ${\mathcal{M}}{\mathcal{A}}^{*}$ correspond to histories that also satisfy $(\mathsf{turn}{=}{a})$ in $\mathcal{G}$ (provided they start in $w_{\iota}$), and we are done.

5 Perspectives

We have incrementally extended the framework of epistemic planning to a game setting where players’ actions are described by action models from DEL. We have established fine-grained results depending on the type of action models.

Works on classical planning that consider game features exist, and they can easily be located in the landscape of decision problems we have considered. Typically, our controller (resp. distributed strategy) synthesis problem subsumes conditional planning with full (resp. partial) observability [34]. Also, conformant planning (partial information where the plan is a sequence of actions) corresponds to a particular case of our distributed strategy synthesis problem where $\textit{Agt}_{\exists}={\{\exists\}}$, $\textit{Agt}_{\forall}={\{\forall\}}$ are singletons, and $\exists$ is blind, i.e., all actions in $A_{\forall}$ are indistinguishable for her. Blindness and uniformity assumption make that the strategies of $\exists$ can be seen as sequential plans.

Moreover, the decision problems we have considered go well beyond classical planning by addressing, e.g. distributed planning with cooperative or/and adversarial features. We are thus confronted in a DEL setting to issues usually met in game theory, as witnessed by the undecidability of the distributed strategy synthesis problem for rather simple action models (Theorem 4.3). However, the DEL perspective we propose offers a language to specify actions (preconditions, postconditions, and epistemic relations between actions) that may help identifying yet unknown decidable cases.

We note that our results should transfer to Game Description Language, equivalent to DEL [18].

One interesting extension of the unifying setting of DEL games would be to consider concurrent games, where players execute actions concurrently, but this will require to first generalise the product operation of DEL. Another direction would be to consider richer objectives such as ones expressed in epistemic temporal logic, which can express not only reachability but also safety or liveness objectives for instance.

Our approach contributes to putting closer the field of multi-agent planning in AI with the field of multi-player games in formal methods. The setting of DEL games may be beneficial to both, allowing the transfer of powerful automata and game techniques from formal methods to epistemic planning, and bringing in multiplayer games new insights from the fine modelling offered by DEL.