A Gentle Introduction to Epistemic Planning: The DEL Approach
Thomas Bolander
TL;DR
This paper provides an accessible introduction to epistemic planning using Dynamic Epistemic Logic (DEL), illustrating how it extends classical planning frameworks like STRIPS to handle multi-agent knowledge and capabilities.
Contribution
It offers a clear, step-by-step explanation of DEL-based epistemic planning, bridging classical planning and complex multi-agent scenarios.
Findings
DEL provides a natural framework for multi-agent epistemic planning
The paper demonstrates how to extend STRIPS with DEL concepts
It facilitates modeling of complex knowledge and capability scenarios
Abstract
Epistemic planning can be used for decision making in multi-agent situations with distributed knowledge and capabilities. Dynamic Epistemic Logic (DEL) has been shown to provide a very natural and expressive framework for epistemic planning. In this paper, we aim to give an accessible introduction to DEL-based epistemic planning. The paper starts with the most classical framework for planning, STRIPS, and then moves towards epistemic planning in a number of smaller steps, where each step is motivated by the need to be able to model more complex planning scenarios.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
A Gentle Introduction to Epistemic Planning:
The DEL Approach
Thomas Bolander DTU Compute
Technical University of Denmark
Copenhagen, Denmark [email protected]
Abstract
Epistemic planning can be used for decision making in multi-agent situations with distributed knowledge and capabilities. Dynamic Epistemic Logic (DEL) has been shown to provide a very natural and expressive framework for epistemic planning. In this paper, we aim to give an accessible introduction to DEL-based epistemic planning. The paper starts with the most classical framework for planning, STRIPS, and then moves towards epistemic planning in a number of smaller steps, where each step is motivated by the need to be able to model more complex planning scenarios.
1 Introduction
Automated planning is a branch of artificial intelligence concerned with computing plans (sequences of actions) leading to some desired goal. A human or robot could e.g. have the goal of picking up a parcel at the post office, and then the problem becomes to find a succesful sequence of actions achieving this. Epistemic planning is the enrichment of planning with epistemic notions, that is, knowledge and beliefs. The human or robot might have to reason about epistemic aspects such as: Do I know at which post office the parcel is? If not, who would be relevant to ask? Maybe the parcel is a birthday present for my daughter, and I want to ensure that she doesn’t get to know, and have to plan my actions accordingly (make sure she doesn’t see me with the parcel). The epistemic notions are usually formalised using an epistemic logic. Epistemic planning can naturally be seen as the combination of automated planning with epistemic logic, relying on ideas, concepts and solutions from both areas.
In general, epistemic planning considers the following problem: Given my current state of knowledge, and a desirable state of knowledge, how do I get from one to the other? It is of central importance in settings where agents need to be able to reason about their own lack of knowledge, and e.g. make plans of how to achieve the required knowledge. It is also essential in multi-agent planning, where succesful coordination and collaboration can only be expected if agents are able to reason about the knowledge, uncertainty and capabilities of the other agents.
In this gentle introduction to epistemic planning, the focus will be on the DEL approach: Using Dynamic Epistemic Logic (DEL) as the underlying formalism. We start with the most classical framework for planning, STRIPS, and then stepwise we expand and generalise the framework until finally reaching full multi-agent planning based on dynamic epistemic logic. Each of these steps will be based on the need to be able to formalise specific planning scenarios.
In Section 2 we will first introduce classical planning domains and planning tasks. Then we move to present the basics of the STRIPS planning framework in Section 3, and propositional planning in Section 4. We then slowly progress towards defining the epistemic planning framework via first defining belief states and conditional actions in Section 5, and then epistemic logic and dynamic epistemic logic in Sections 6–7. In Section 8 we finally define epistemic planning tasks, and study various extensions and generalisations in Sections 9–11. In Section 12 we very briefly study complexity issues of epistemic planning, and we round off with a discussion of alternative approaches to epistemic planning in Section 13.
2 Classical planning domains and planning tasks
Example 1**.**
Suppose a father has his daughter’s birthday coming up, and he ordered a present for her which is now at the post office. His goal is to be able to give her the present the next day. This is a planning task (sometimes called a planning problem): He needs to compute a plan to achieve the goal. In a planning task, one is given an initial state, a set of goal states and a set of available actions. The problem is now to compute a sequence of actions (a plan) that, if executed in the initial state, will lead to one of the goal states. In the birthday present example, the initial state describes that the present is at the post office and not yet wrapped. The goal states are those in which the present is at home and wrapped (ready to be given on the next day). The available actions could be actions like going from home to the post office, going from the post office to home, picking up the present at the post office, and wrapping the present. Of course we do not need to limit ourselves to only allowing these specific actions, but could have general actions for going from a location to a location , general actions for picking up an object at a location, and general actions for wrapping an object that you are currently holding.
To allow us to reason formally about planning tasks and plans, and to allow computers and robots to compute plans, we need an appropriate formalism to describe such objects. The simplest approach is to define planning tasks in terms of state-transition systems.
Definition 1**.**
[18] A (restricted) state-transition system (also called a classical planning domain or simply a state space) is where:
- •
is a finite or recursively enumerable set of states.
- •
is a finite or recursively enumerable set of actions.
- •
is a computable partial state-transition function.
When is defined, is said to be applicable in . When is a sequence of actions from , we write for .111So is the result of executing the actions of in sequence starting in .
Definition 2**.**
[18] A classical planning task is a triple where:
- •
is a state-transition system (a classical planning domain).
- •
is the initial state.
- •
is the set of goal states.
A solution to a classical planning task is a finite sequence of actions (a plan) from such that . The length of a solution is the number of actions in .
Example 2**.**
Consider the birthday present example from Example 1. The available actions and their corresponding state transitions could be presented by the state-transitions system of Figure 1. Here we have with , , and is as given in the figure (e.g. since there is an edge labelled “go to post office” from to ). Note that “wrap present” is only applicable after “pick up present” has been executed: it is necessary to get hold of the present before it can be wrapped. The planning task of Example 1 can then be represented as the classical planning task where and . A solution is highlighted in Figure 1:
[TABLE]
Representing planning tasks directly as state-transition systems has some important weaknesses: 1) there is no internal structure on states and actions to understand or clarify what they represent; 2) state-transition systems are normally of exponential size in the number of propositional variables required to describe the states. To exemplify the second weakness, if the planning task was to take home parcels from the post office, the state space (state-transition system) would be of size : each parcel could either be at the post office or at home. This is so even though the length of the shortest solution would only be linear in (e.g. bring the parcels home one at a time). To address both of these weaknesses one introduces logical structures on states and actions. This is e.g. done in the planning language STRIPS to be introduced next.
3 STRIPS planning
The classical language for describing states and actions in the field of automated planning is STRIPS [17]. We will here introduce STRIPS slightly informally, and the interested reader is referred to [30] for more details. The reader familiar with STRIPS planning can skip this section, but might want to briefly look at the examples.
In STRIPS, states are represented as sets of ground atoms of a function-free first-order language . In the birthday present example, the initial state could e.g. be described by
[TABLE]
where is a predicate for expressing that object (or agent) is at location . The predicates , and are true of agents, objects and locations, respectively (alternatively, one could use a sorted first-order language and then omit these predicates). Actions are described via so-called action schemas. The actions of going from one location to another, picking up an object and wrapping an object can be expressed by the STRIPS action schemas provided in Figure 2, where predicates have names starting with upper-case letters, and variables have names starting with lower-case letters. Each action schema has a name, a precondition and an effect. Preconditions and effects are conjunctions of literals of the first-order language . A ground action is achieved by instantiating all variables of an action schema with constants of . For instance,
[TABLE]
The precondition of a ground action describes what has to be true for the action to be applicable.222More formally, a ground action is applicable in a state if where denotes semantical entailment on propositional logic over the set of ground atoms of . It is easy to check that is applicable in the initial state defined by (1) (all precondition atoms of the action occur in ). The effect of a ground action describes how the state is modified when the action is executed. The effect of expresses that becomes true, and that becomes false. Hence the result of executing in will be the state
[TABLE]
Any finite set of STRIPS action schemas induce a state-transition system (classical planning domain) by:
- •
, where is the set of ground atoms of .
- •
A^{\prime}=\{\text{all ground instances of the action schemas in A}\}
- •
\gamma(s,a)=\begin{cases}(s-\{\varphi\mid\neg\varphi\text{ is a negative literal of }\textsc{Effect}(a))\ \cup\\ \quad\{\varphi\mid\varphi\text{ is a positive literal of }\textsc{Effect}(a)\}&\text{if s\models\textsc{Precond}(a)}\\ \text{undefined}&\text{otherwise}\end{cases}
The state-transition system induced by the STRIPS schemas of Figure 2 is provided in Figure 3. The rigid atoms, those that cannot change truth-value, have been omitted. Furthermore, each constant name is abbreviated by the capital letters it contains (so for instance PostOffice is abbreviated PO).
Note that the resulting state-transition system is isomorphic to the state-transition system of Figure 1.
The advantage of the current STRIPS representation over the previous pure state-transition representation is that there is now structure on states and actions. This for instance means that the current formulation is easily generalisable, e.g. we can add additional parcels to the initial state without any need to modify the underlying action schemas. The induced state-transition system would become exponentially bigger, as earlier noted, but the size of the STRIPS action schemas would stay the same (though the size of the description of the initial state would grow linearly with ).
A STRIPS planning task on a function-free first-order language is where is a set of STRIPS action schemas over , is a set of ground atoms over and is a conjunction of ground literals over called the goal formula. Any STRIPS planning task induces a classical planning task by letting be the state-transition system induced by and letting . A solution to a STRIPS planning task is then a solution to the induced classical planning task.
Example 3**.**
The birthday present example can be represented as a STRIPS planning task where is the set of action schemas of Figure 2, is defined by (1), and . This is a STRIPS planning task on the function-free first order language with binary predicate symbols At and Has, unary predicate symbols Wrapped, IsAgent, IsLocation and IsObject, and constant symbols Home, PostOffice, Father, Present. Consulting Figure 3, it is easy to show that a solution to this planning task is
[TABLE]
In the field of automated planning, actions are always described compactly in an action description language like STRIPS (or e.g. PDDL, ADL or SAS+). A lot of research effort goes into finding ways to automatically derive efficient heuristics from action schemas, such that solutions can be found with minimal search. If given an induced state-transition system of a set of STRIPS action schemas, finding a solution to a planning task becomes a simple graph search problem (find a path from to a state in ). This can be done in linear time in the size of the state-transition system. However, as earlier noted, the induced state-transition system is often exponential in the size of the action schemas. Hence the complexity of computing solutions or deciding whether a solution exists (the plan existence problem) is in automated planning always measured in the size of the compact action schema representation. This is different from many formalisms in logic that consider plans or strategies and where complexity is measured in terms of the size of the state space. This is e.g. why epistemic planning based on ATEL in [20] can be claimed to be tractable, even though already basic propositional STRIPS planning, which is much less expressive, is intractable. For most planning domains considered in automated planning (e.g. the planning domains of the International Planning Competition, IPC), calculating the entire state-transition system is computationally infeasible, and the goal is then for the heuristics to be sufficiently efficient that only the most relevant parts of the state-transition system are explored.
4 Propositional planning tasks
Even though STRIPS action schemas are formulated using first-order logic, for most purposes we can consider STRIPS as a planning formalism based on propositional logic. To see this, we first define propositional planning tasks.
Definition 3**.**
A propositional planning task333Called a set-theoretic planning task in [18]. on a finite set of atomic propositions is where
- •
is a finite set of actions. Each action is a pair where and are conjunctions of propositional literals over . The element is called the precondition of and its postcondition.
- •
is the initial state, a propositional state over (a subset of ).444In general, we will identify propositional states with subsets of . A subset represents the propositional state in which the elements of are the atomic propositions true in .
- •
is the goal formula, a propositional formula over .
A propositional planning task on induces a classical planning task in the expected way (compare with the classical planning task induced by a STRIPS planning task defined in Section 3):
- •
- •
\gamma(s,a)=\begin{cases}(s-\{p\mid\neg p\text{ is a negative literal of }post(a))\ \cup\\ \quad\{p\mid p\text{ is a positive literal of }post(a)\}&\text{if s\models pre(a)}\\ \text{undefined}&\text{otherwise}\end{cases}
- •
.
A solution to a propositional planning task is any solution to the induced classical planning task.
For any function-free first-order language , let denote the set of ground atoms of . The set can be thought of as the atomic propositions of a propositional language. Any quantifier-free ground formula of is then at the same time a formula of propositional logic over . Any STRIPS planning task on induces a propositional planning task on by simply letting
[TABLE]
It is easy to show that the STRIPS planning task and its induced propositional planning task both induce the same classical planning task. They thus also have the same solutions. Note that the conventions for preconditions and effects are a bit different in propositional planning tasks than in STRIPS planning tasks. We now write precondition and effect pairs of an action as a pair of the form , where we have also relabelled effects as postconditions. The point of both these changes is to gradually move away from the classical conventions of STRIPS planning into the conventions of dynamic epistemic logic that we will later present a planning framework based on.
Example 4**.**
The birthday present example can be represented as the propositional planning task below. It is a simplified version of the propositional planning task induced by the STRIPS planning task of Example 3, where we have done away with the rigid atoms that are no longer necessary. In the definitions below, Agt is a set of agent names (including Father), Loc is a set of locations (including Home and PostOffice) and Obj is a set of objects (including Present).
- •
where, for all , all and all ,
- –
- –
- –
.
- •
.
- •
.
Note that expressions like with and are no longer considered as ground atoms of the original first-order language , but as atoms of propositional logic over . A solution to this planning task is exactly as to the original STRIPS version: .
Since any STRIPS planning task can be propositionalised as above, it means we can now work in a simpler formalism, propositional logic, which also makes it easier to generalise the formalism to e.g. planning with partial observability, non-determinism or epistemic planning.555In certain cases the difference between expressing a planning task in the lifted first-order STRIPS representation and its induced propositionalisation becomes essential: for some of the complexity results measured in the size of the planning tasks; for practical convenience of representation; or e.g. for learning actions/action models [31, 25]. However, for the purposes of this paper, the grounded/propositionalised representation is sufficient.
5 Belief states, partial observability, and conditional actions
Consider the birthday present example formalised as the propositional planning task of Example 4. Assume now that there is not only one, but two, local post offices, and the father does not know in which one the parcel is. We can assume Loc correspondingly contains two constants, PostOffice1 and PostOffice2. To represent this modified planning task we need two changes in the underlying formalism: belief states and conditional actions. We need belief states to represent the uncertainty of the agent. A belief state in this setting is a set of propositional states, that is, a set of subsets of (where is the set of atomic propositions). The initial belief state of our agent is now:
[TABLE]
The state contains two propositional states, where the first one represents the situation where the present is in PostOffice1, and the second presents the situation where it is in PostOffice2. We define a propositional formula to be true in a belief state , written , if is true in all propositional states of . Hence we have
- (1)
2. (2)
3. (3)
4. (4)
.
This represents the internal perspective of the father in the belief state : He can verify (knows) that he is home (1) and can verify (knows) that the present is in PostOffice1 or PostOffice2 (4), but doesn’t know which (2–3). Planning in the space of belief states rather than propositional states is called planning under partial observability (and planning on propositional states is then called planning under full observability). In the following, and in line with modal logic, we will call the elements of belief states worlds.
To represent the modified example we also need to allow conditional actions. The agent can attempt to pick up the present at either of the two post offices, but whether he is succesful is conditional on whether it is the correct one. Symmetric to the generalisation from representing states as subsets of to sets of such subsets, we can generalise actions from being pairs to be sets of such pairs. We can then represent the attempted pickup action by:
[TABLE]
where the postcondition means that nothing changes. From now on we will, in line with the literature on dynamic epistemic logic, call pairs events. So a conditional action like the one above is a set of events: a set of the possible things that might happen when the action is executed. The TryPickUp action above expresses that if the agent and the object are in the same location, the object will be successfully picked up (the first event of the action), and otherwise nothing will happen (the second event of the action).
Given a belief state represented as a set of worlds and an action action represented as a set of events, we can define a generalised transition function by
[TABLE]
Thus, e.g., where we abbreviate PostOffice1 by PO1, PostOffice2 by PO2 and Home by H:
[TABLE]
[TABLE]
Note how the belief state goes down from cardinality 2 to cardinality 1 when going from to . At plan time, when deliberating about the possible action sequences, the father doesn’t know whether he will have the present after having visited only PostOffice1, and he hence has to represent both possibilities. In , however, he knows, even at plan time, that he will have the present, since if he didn’t get it at PostOffice1, he will be sure to get it at PostOffice2. The calculations above show that a solution to the modified planning task is
[TABLE]
There are, unfortunately, several weaknesses in the current approach. An important weakness is that the formalism does so far not distinguish between between the kind of uncertainty the father has in and the kind of uncertainty he has in . In , the father has run time uncertainty [27] about the location of the present. This means that even at execution time, when the action has been executed in the initial state, the father still does not know which of the two worlds of is the actual one. In , however, he should only have plan time uncertainty [27]. This means that when he is deliberating about the possible action sequences and their potential outcomes, he can not tell which of the two worlds of is going to be realised, but at execution time he will know. This is because attempting to pick up the present at PO1 has the side-effect of informing him whether the present is there or not. The distinction between plan time and run time uncertainty is not represented in our current formalism, since both kinds of uncertainty are modelled exactly the same: by simply having a set of worlds representing those situations that the agent cannot distinguish between (whether at plan time or at run time).
The simplest and most common solution to this problem in the automated planning literature (see e.g. [18, 29]) is to treat observability as a static partition on the set of possible worlds, so that certain possible worlds are always distinguishable and others never are. What becomes distinguishable at execution time is then hardcoded into this observability partition. This approach is however insufficient for our purposes. For instance, assume the present is initially located at PO2. Assume further that the father initially goes to PO1 to attempt picking it up and then goes home again. Then afterwards he will be in exactly the same world as initially, namely the world satisfying . But he will not be in the same information state: He learned that the present is not at PO1 (and therefore also learned that it must be at PO2). To be able to represent this in an appropriate way we need to take the final step into planning based on epistemic logic.777In principle, in the single-agent case, we could alternatively consider modelling states as sets of sets of worlds (that is, elements of ) and actions a sets of sets of events. In a state , two worlds and would then always be plan time indistinguishable, but only run time indistinguishable if . This is essentially equivalent to representing states as models of single-agent epistemic logic (see next section). Since we are anyway going to generalise to multi-agent planning, we choose to move straight to epistemic logic, rather than consider this further intermediate step.
6 Epistemic logic
Let be a finite set of atomic propositions (often we will take to be the set of ground atoms of a function-free first-order language as in the previous sections). Let be a finite set of agents. The epistemic language on , denoted , is generated by the following BNF:
[TABLE]
where and . We read as “agent knows ” and as “it is common knowledge that ”. When and are clear from the context (or no assumptions about them are made), we will write for . is the language without the modality.
Definition 4**.**
Let and be as above. An epistemic model on is where
- •
The domain is a non-empty finite set of worlds.
- •
is an equivalence relation called the indistinguishability relation for agent .888We will later consider generalising to non-equivalence relations, but for now it is sufficient ot make this simplification and only consider equivalence relations.
- •
is a labelling function assigning a propositional valuation (a set of propositions) to each world.
For , the pair is called an epistemic state (or simply a state), and the worlds of are called the designated worlds. A state is called global if for some world (called the actual world), and we then often write instead of . We use to denote the set of global states (or simply if and are clear from the context). For any state we let . A state is called a local state for agent if is closed under (that is, if and implies ). Given a state , the associated local state of agent , denoted , is .
Definition 5**.**
Let be a state on with . For , and , we define truth as follows:
[TABLE]
where is the transitive closure of .
Note that a formula is defined to be true in a non-global state iff it is true in each of the global states , , it contains. This is consistent with our previous definition of truth in belief states as truth in all worlds of that state.
Example 5**.**
Let , where is the function-free first-order language of Example 3 extended with constant symbols PostOffice1 and PostOffice2 and let . Consider the initial state defined by (2). We can represent this via an epistemic model on with , , and . Graphically, this epistemic model is represented as follows, using the abbreviations from earlier:
[TABLE]
Here the nodes represent the worlds and the edges represent the indistinguishability relation (reflexive edges left out). Each node is labelled by its name and the set of atomic propositions true at the world. Consider the case where initially the present is at PostOffice2. This means that the actual world is , and this situation can be represented by the global state , that we graphically present as
[TABLE]
We use for designated worlds. The planning agent, Father, does not have access to this global state, since he considers it equally possible that the present is at PO1 ( and are indistinguishable to him). The internal initial state of the father is his associated local state :
[TABLE]
We have but : From the outside global perspective (the perspective of an omniscient external observer) it can initially be verified that the present is at PO2, but not from the perspective of the planning agent, Father, himself.
In general, any belief state (set of propositional states) can be equivalently represented as an epistemic state with , , for all .
With epistemic models we can also capture the distinction between run time and plan time uncertainty. When the father contemplates on the possible action sequences to execute in his local state , the epistemic state representing the result of executing the action sequence
[TABLE]
will be
[TABLE]
There is still two designated worlds: the agent do not at plan time know which one it will end up in. But they are now not indistinguishable (there is no indistinguishability edge connecting them): at run time, when the plan is executed, the agent will be able to tell whether it is in or . In fact we have e.g. : In , the father knows whether the present is at PO2 or not. More generally, let be any local state of an agent and let . The worlds and are called run time indistinguishable (for agent ) if . Otherwise, they are plan time indistinguishable.
The generalisation from belief states (in the standard AI sense of sets of propositional states [30]) to epistemic states (in the sense of epistemic logic) also allows us to represent planning tasks involving multiple agents. E.g. the father might want to make sure that his daughter doesn’t get to know that he has a present for her (it is supposed to be a surprise). In this case, we could reformulate the goal as
[TABLE]
7 Dynamic epistemic logic
Generalising from propositional states to epistemic states amounted to generalise from propositional valuations to multisets of such valuations with an added indistinguishability relation for each agent. We can now apply the exact same trick to generalise from propositional actions to epistemic actions. A propositional action is an event , so an epistemic action will be a multiset of events with an indistinguishability relation for each agent. This is exactly how epistemic actions are defined in dynamic epistemic logic (DEL) with postconditions [13]. Such structures are called event models or action models.
Definition 6**.**
An action model on is where
- •
The domain is a non-empty finite set of events.
- •
is an equivalence relation called the indistinguishability relation for agent .
- •
assigns a precondition to each event.
- •
assigns a postcondition to each event. For all , is a conjunction of literals over (including the special atom ).999This definition of postconditions is slightly less general than the standard definition of postconditions in DEL [13]. Normally, is a mapping from atomic propositions to formulas , where means that after the event has occurred gets whatever truth value had before occurred. In [13] it is shown that any action model can be equivalently represented by one in which for all and . This action model can potentially be exponentially larger. Such simplified (but potentially larger) action models can immediately be translated into the form defined here (see [7] for details). The advantages of the conventions of this paper is to have a more clear connection to the conventions of classical planning and to simplify notation.
For , the pair is called an epistemic action (or simply an action), and the events in are called the designated events. An action is called global if for some event , and we then often write instead of . Similar to states, is called a local action for agent when is closed under . Given an action , the associated local action of agent , denoted , is (\mathcal{E},\{f\in E\mid\text{f\sim_{_}iee\in E_{_}d}\}).
Any propositional action trivially induces an epistemic action with where and .
Example 6**.**
We can now finally, using action models, give a satisfactory formal representation of the TryPickUp action of Section 5, cf. (3):
[TABLE]
We here use the same conventions for representing action models graphically as we did for epistemic models. Note that there is no indistinguishability edge for Father: He will at run time observe whether the action is succesful () or not (). But at plan time he does not know which one it will be, which is why both events are designated.
The state-transition function of DEL is called product update, and is denoted by an infix symbol. So instead of writing , one writes .
Definition 7**.**
Let a state and an action be given with and . Then the product update of with is where
- •
- •
- •
- •
.
We say that an action is applicable in a state if for all there is an event such that .
Note that this is the obvious generalisation to the epistemic setting of the state-transition function for belief states and sets of events given in (4). In particular, if is the epistemic state induced by a belief state , and is the action model induced by a propositional action , then is the epistemic state induced by .
Let be an agent, let be a local state for and let be a local action for . Then and represent agent ’s view on a particular state and action. According to the definition above, the action is then applicable in the state if, for any of the worlds agent considers possible (any world in ), the action specifies at least one applicable event that agent considers possible (an event in with ). In other words, the action is applicable from the perspective of agent .
8 Epistemic planning tasks
We now have all the necessary ingredients for defining planning tasks in epistemic planning based on DEL. We simply define these as for propositional planning tasks, except we replace propositional actions by epistemic actions, propositional states by epistemic states and propositional goal formulas by epistemic goal formulas.
Definition 8**.**
An epistemic planning task on is where is a finite set of epistemic actions on , is an epistemic state on , and . We say that is an epistemic planning task for agent if and all are local for . A planning task is called global if is global. Given any planning task , the associated local planning task of agent , denoted , is .101010Given a planning task , the planning task is agent ’s local perspective on that planning task. E.g. if the present is at PO1, then from an outside perspective, the planning task has an initial state where only the world satisfying is designated, but agent Father is still forced to do planning based on the planning task where both the -world and the -world are designated, since the father doesn’t know which of the two is the actual initial state.
An epistemic planning task induces a classical planning task in a similar manner to propositional planning tasks:
- •
- •
\gamma(s,a)=\begin{cases}s\otimes a&\text{if as}\\ \text{undefined}&\text{otherwise}\end{cases}
- •
.
As for propositional planning tasks, we can then define a solution to an epistemic planning task to be a solution to the induced classical planning task. This is equivalent to the existence of a sequence of actions from such that each is applicable in () and .
Example 7**.**
Let us provide a full formalisation of the birthday present example as an epistemic planning task. The epistemic action TryPickUp was presented in Example 6. The actions and are simply the ones induced by their propositional counterparts. The initial state is the state of Example 5. The goal formula is . Clearly the resulting planning task is a planning task for agent Father. We now get
[TABLE]
Note that the indistinguishability link between the two worlds is being cut when going from to , since the two events of TryPickUp are distinguishable. We further get
[TABLE]
Since the two worlds of have identical labels, we can take the bisimulation contraction which will preserve only one of them.111111For a definition of bisimulation between epistemic models with multiple designated points, see [7]. For a definition of bisimulation on models of modal logic in general, and a definition of bisimulation contraction, see [5]. Bisimulation contractions are an indispensible tool in practice when working with product updates in DEL, since sequences of updates otherwise tend to produce very large model. We clearly have . Hence the action sequence (5) above is a solution to the epistemic planning task. We could actually replace the action in this plan with the epistemic action induced by the original non-conditional action. This is because the father will know that the present is at PO2 when he gets there (since it wasn’t at PO1).
9 Conditional epistemic planning
The plan found in the previous example, Example 7, is clearly not always optimal. If the father gets the present already at PO1, there is no need to go to PO2 as well. So in this case the father will be able to reach the goal in 4 instead of 6 steps. But the father will of course not know this until run time (after having attempted to pick up the present at PO1). A sequential plan (sequence of actions) can not capture this, since it has a fixed number of actions, independent of the action outcomes. So we need to move to conditional plans (not to be confused with conditional actions that we already have). A conditional plan for the current planning task could e.g. be formulated as the following program
[TABLE]
Such programs for epistemic planning tasks are formally defined and explored in [2] (single-agent case only).121212Note that the if-condition of the if-then-else construct is a -formula expressing knowledge of the planning agent (Father). All such if-conditions are required to be -formulas of the planning agent, as an agent can only make a choice based on what he knows. Otherwise we could simply construct a conditional plan like . However, in this case the agent would not know how to settle the truth-value of the if-condition in the initial state, and would hence not know whether to execute or . Alternatively, such programs can be seen as abbreviations of PDL programs, e.g. the program can be seen as shorthand for the PDL program . In [14], dynamic epistemic logic with postconditions and PDL constructs is studied (however not in a planning context). Programs like the one above are in [16] called knowledge-based programs and are there studied in an alternative logical setting. Knowledge-based programs are studied in a planning context in [24].
An alternative to conditional plans as programs is policies. A policy is a mapping from states into actions, that is, for each state the policy specifies the action chosen in that state (a policy is also sometimes called a strategy or a protocol, depending on the area). Since implemented planning systems often generate conditional plans via an And-Or graph search in the state space, a policy comes as a more direct output of these algorithms than a program (e.g. one does not have to synthesise appropriate if-conditions). From now on we will only consider conditional plans as policies, not programs.
Definition 9**.**
Let be a planning task and be an agent. An -policy for is a partial mapping from global states into actions such that
- (1)
If then is applicable in . 2. (2)
If then .
An -policy is a policy from the perspective of agent . Condition (1) ensures that in such a policy, agent always knows that the next action to be performed is applicable (knowledge of preconditions). Condition (2) is a uniformity condition: If two global states are indistinguishable to agent , agent has to make the same choice in both.
Definition 10**.**
An execution of a policy from a global state is a maximal (finite or infinite) sequence of alternating global states and actions such that for all ,
- (1)
, and 2. (2)
.
An execution is called successful for a global planning task if it is a finite execution such that .
Definition 11**.**
A policy for a planning task is called a solution to if and for each , any execution of from is successful for .131313Note that a solution policy has to lead to successful executions for all global states in (and for all global states along all possible executions of the policy). Such policies are often called strong policies, because the requirement is that they are guaranteed to succeed under alle circumstances (considered possible by the planning agent). In conditional planning, one often also considers weak solutions (weak policies) that have a possibility of succeeding, but might not always succeed. A weak solution to the birthday present task would be to go to PO1, attempt picking up the present there, go home, and then wrap the present if one has it. This plan can clearly fail if the present is at PO2. In this paper we restrict attention to strong solutions.
Example 8**.**
In the birthday present task we can now specify distinct actions to be performed in the two distinct global states of the state of Example 7 (the state after the father has attempted to pickup the present at PO1):
[TABLE]
This partial policy can easily be extended into a full solution policy to the planning task.
Example 9**.**
Consider extending the birthday present example with a post office employee, Employee, working at PO1. We can think of this as a new agent, so now . It might be possible to call the employee from home to ask whether the present is at the post office. We can assume the employee knows whether the present is at his or her post office, so the initial state would be the same as in Example 7 (), except there is now reflexive loops for Employee at both worlds. We can now represent a general action of an agent calling an agent to ask whether a formula is true as the following action, where and :
[TABLE]
The action model above corresponds to agent giving a sincere answer to the question “is true?”. Agent saying “yes” to the question is represented by the event , that is, it corresponds to an announcement that agent knows . Event corresponds to saying “no”, and corresponds to saying “I don’t know”. We now get
[TABLE]
This is because in of we have that holds and in of we have , and the action model for is clearly seen to cut the link between worlds satisfying and worlds satisfying (since there are two distinguishable events with these formulas as preconditions). Hence the planning agent, Father, can conclude
[TABLE]
and hence that he can first call to ask the post office employee whether she has the present, and then next he will know where to go.
10 Public and private actions
The Ask action above is a bit simplified. It is publicly observable: the indistinguishability relation is the identity. When there is only two agents in the scenario this is acceptable, since they can only call each other, but if there were more agents present, these would probably not observe the phone call taking place. Consider adding a third agent, Employee2, working at PostOffice2. If Father calls Employee, Employee2 will not know, and might not even consider it possible that the phone call could have taken place. To model this we have to go beyond equivalence classes in epistemic models and action models, since now it becomes possible to have false beliefs, e.g. a false belief that no phone call took place (or a false belief that Father does not yet know ). In this case, we could model the private phone call of agent to agent asking about as follows:
[TABLE]
This models that agent and know the phone call takes place (and know the outcome of the phone call), but all other agents think that nothing has happened (the skip event ). We can even model that some subset of agents hear the question being asked, but not the answer (they are together with , but cannot hear what is being said in the other end):
[TABLE]
These examples illustrate that designing action models to represent actions under multi-agent partial observability is far from a trivial matter. Some recent papers try to propose different ways of encoding observability information into a logical language, so that the action models themselves can become simpler, more uniform, and automatically induced from the underlying observability information [6, 8, 19]. However, there are still many unsolved problems in this area.
The possibility of representing private actions (only observed by a subset of agents, whereas the other agents believe nothing happens) is also what allows us to model the full version of the birthday present example, where the daughter does not get to know that the father has the present. We can for instance model that only agents who are copresent (in the same location) can observe actions taking place in that location (see [11] for a discussion and treatment of copresence in an epistemic planning setting). For instance, the Wrap action could be reformulated as follows:
[TABLE]
We here use a new type of action model that we call an edge-conditioned action model [6]. The meaning of the label on the edge is that there is an edge for agent here if is true (agent is not present in the location where the present is being wrapped). Such edge-conditioned action models are a variant of generalised arrow updates [23].
With the wrapping action presented as above, the father will e.g. be able to figure out that if the daughter is at home ( is true), then only the plan where he executes Wrap at the post office will lead to (recall the state-transition system in Figure 2 that shows that the present can either be wrapped in the post office or at home).
11 Multi-agent planning
So far we have only considered the case of a single agent planning in the presence of other agents. The formal framework of course also allows to represent real multi-agent planning tasks. However, for true multi-agent planning a lot of additional design choices have to be made. Are the agents collaborating or competing? Are there coalitions of agents trying to achieve a joint goal, perhaps against the agents outside the coalition? Can the agents communicate and coordinate arbitrarily when coming up with a plan, and will they commit to such a joint plan? The papers [15, 9] look at implicitly coordinated plans, where agents have a joint goal but are not allowed to coordinate or negotiate in advance. All coordination should happen during plan executions as a result of announcements and observing the actions of others.
In the birthday present example it could e.g. be that Father calls Employee to announce the goal . Assuming Employee is altruistic and adopts this as his or her own goal, Father and Employee are now engaged in planning with implicit coordination towards the joint goal . Assume the present is at PostOffice2. Since Employee knows that Father doesn’t know the location of the present, and that he cannot get the present unless he knows, she will choose to announce that the present is at PostOffice2. This is a case of implicit coordination: The father doesn’t ask the employee about the location of the present, but the employee knows about the goal and can still plan to inform him, in order to allow him to plan the rest of the required actions. These ideas are explored further in [15, 9].
12 (Un)decidability and complexity
One of the most studied problems in epistemic planning based on DEL is the complexity of the plan existence problems. The plan existence problem for a class of planning tasks is the following decision problem: Given a planning task , does there exist a solution to ? So far only the complexity of deciding whether a sequential plan exists have been studied. For general epistemic planning tasks the problem is undecidable already with two agents, no common knowledge, and even without postconditions (that is, purely epistemic planning without ontic change) [3]. This has lead to a quest for finding decidable, but still reasonably expressive, fragments of epistemic planning. The first result along these lines proved that epistemic planning with propositional preconditions (that is, no epistemic formulas in the preconditions) is decidable [32]. An upper bound on complexity is in that paper shown to be -ExpTime for planning tasks in which the goal formula has modal depth . In [12] it is shown that when the preconditions are propositional and there are no postconditions, then the plan existence problem is PSpace-complete (for arbitrary goal formulas). If restricting further to certain types of actions, like private and public announcements, complexity of plan existence goes further down to NP-complete [10].
13 Alternative approaches to epistemic planning
Epistemic planning based on DEL takes a semantic approach, where states are represented as semantical objects, epistemic states. It is also possible to take a syntactic approach, where states are represented as knowledge-bases, sets of formulas known to be true. Interestingly, STRIPS and propositional planning are semantic and syntactic at the same time, since in propositional logic a semantic state is just a set of formulas (a set of true atomic propositions). This means that when generalising from STRIPS or propositional planning to epistemic planning, it is not immediately obvious whether a semantic or syntactic approach would be most appropriate. Both approaches have their strength and weaknesses.
Epistemic planning based on DEL also takes the approach of insisting on succinct action representations via action models. We argued in favour of this approach already very early in the paper by mentioning the weaknesses of attempting to represent planning tasks explicitly as state-transition systems. In fact, the state-transition systems induced by epistemic planning tasks will often be infinite, so even if choosing a state-transition system based approach, one will need a way to represent them finitely in order to be able to do planning based on them [21]. From the perspective of classical planning, the most appropriate approach to representing state-transition systems compactly seems to be representing the actions themselves in a compact way (using some kind of action models or action schemas).
Based on the above, we can distinguish approaches to epistemic planning along two dimensions:
- •
Semantic approaches versus syntactic approaches.
- •
Action model based approaches versus state-transition system based approaches.
The approach of this paper is, as mentioned, semantic and action model based. Syntactic approaches to epistemic planning can be found in e.g. the (single-agent) PKS planner [28], the (multi-agent) planning framework of [26] and the compilation approach of [22], translating a restricted fragment of epistemic planning into classical planning. The state-transition system based approach is found in a number of papers in temporal epistemic logic, e.g. [20, 21]. In these papers, the planning domains are represented by a type of epistemic state-transition system called a concurrent epistemic game structure (CEGS). Having an explicitly represented state-transition system like a CEGS tends to make it easier to define complex notions of multi-agent plans (strategies). On the other hand, by working directly with state-transition systems, some of the important problems of epistemic planning are being silently bypassed. These problems include compact representations of the state-transition system, and efficient heuristics for avoiding to build the entire state-transition system when planning. They also include problems of how to provide a general approach to multi-agent observability and problems of concurrent composition of actions of multiple agents. In CEGS, transitions represent joint actions, one action per agent. The problem of how to make a parallel composition of the actions of the individual agents has to be solved before one can even build the CEGS for a given planning domain. In epistemic planning based on DEL, this problem has to be solved at the level of action models: How do we make a parallel composition of two action models in case they are not independent (e.g. two agents trying to open a door at the same time, but from opposite sides). Epistemic planning tasks can be seen as inducing e.g. forests of epistemic temporal logic [4] or CEGSs. The exact relations have not been explored yet, but providing a link between the action model based approaches and the state-transition system based approaches could be very valuable and provide a possibility of getting the best of both worlds.
Similarly, providing links and connections between syntactic and semantic approaches seem to be potentially very valuable. In a semantic approach, one is essentially modelling ignorance: the more ignorance, the bigger the state. In a syntactic approach, one is essentially modelling knowledge: the more knowledge, the bigger the state. Knowledge and ignorance are each others duals, and hence the semantic and the syntactic approach are also each others duals. It seems that when humans are planning, we are sometimes using a more semantic approach, and sometimes a more syntactic one, depending on the planning task at hand. It would hence also be very interesting to see whether planning frameworks could be developed that would employ or combine both approaches.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1]
- 2[2] Mikkel Birkegaard Andersen, Thomas Bolander & Martin Holm Jensen (2012): Conditional Epistemic Planning . Lecture Notes in Artificial Intelligence 7519, pp. 94–106, 10.1007/978-3-642-33353-88 . Proceedings of JELIA 2012. · doi ↗
- 3[3] Guillaume Aucher & Thomas Bolander (2013): Undecidability in Epistemic Planning . In: Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence (IJCAI) , pp. 27–33.
- 4[4] Johan van Benthem, Jelle Gerbrandy & Eric Pacuit (2007): Merging frameworks for interaction: DEL and ETL . In: Proceedings of the 11th conference on Theoretical aspects of rationality and knowledge , TARK ’07, ACM, New York, NY, USA, pp. 72–81, 10.1145/1324249.1324262 . · doi ↗
- 5[5] P. Blackburn, M. de Rijke & Y. Venema (2001): Modal Logic . Cambridge Tracts in Theoretical Computer Science 53, Cambridge University Press, Cambridge, UK, 10.1017/CBO 9781107050884 . · doi ↗
- 6[6] Thomas Bolander (2014): Seeing is Believing: Formalising False-Belief Tasks in Dynamic Epistemic Logic . In Andreas Herzig & Emiliano Lorini, editors: Proceedings of the European Conference on Social Intelligence (ECSI-2014) , CEUR Workshop Proceedings 1283, CEUR-WS.org, pp. 87–107.
- 7[7] Thomas Bolander & Mikkel Birkegaard Andersen (2011): Epistemic Planning for Single- and Multi-Agent Systems . Journal of Applied Non-Classical Logics 21, pp. 9–34, 10.3166/jancl.21.9-34 . · doi ↗
- 8[8] Thomas Bolander, Hans van Ditmarsch, Andreas Herzig, Emiliano Lorini, Pere Pardo & François Schwarzentruber (2015): Announcements to Attentive Agents . Journal of Logic, Language and Information , pp. 1–35, 10.1007/s 10849-015-9234-3 . · doi ↗