Timeline-based Planning and Execution with Uncertainty: Theory, Modeling Methodologies and Practice

TL;DR

This paper presents a comprehensive study of timeline-based planning under uncertainty, introducing formal semantics, hierarchical modeling, and a general framework validated in manufacturing contexts.

Contribution

It offers a formal characterization of timeline-based planning with uncertainty, a hierarchical modeling approach, and a versatile framework for real-world applications.

Findings

Formal semantics for timeline-based planning under uncertainty

Hierarchical modeling improves planning efficiency

Framework successfully applied in manufacturing scenarios

Abstract

Automated Planning is one of the main research field of Artificial Intelligence since its beginnings. Research in Automated Planning aims at developing general reasoners (i.e., planners) capable of automatically solve complex problems. Broadly speaking, planners rely on a general model characterizing the possible states of the world and the actions that can be performed in order to change the status of the world. Given a model and an initial known state, the objective of a planner is to synthesize a set of actions needed to achieve a particular goal state. The classical approach to planning roughly corresponds to the description given above. The timeline-based approach is a particular planning paradigm capable of integrating causal and temporal reasoning within a unified solving process. This approach has been successfully applied in many real-world scenarios although a common interpretation of the related planning concepts is missing. Indeed, there are significant differences among the existing frameworks that apply this technique. Each framework relies on its own interpretation of timeline-based planning and therefore it is not easy to compare these systems. Thus, the objective of this work is to investigate the timeline-based approach to planning by addressing several aspects ranging from the semantics of the related planning concepts to the modeling and solving techniques. Specifically, the main contributions of this PhD work consist of: (i) the proposal of a formal characterization of the timeline-based approach capable of dealing with temporal uncertainty; (ii) the proposal of a hierarchical modeling and solving approach; (iii) the development of a general purpose framework for planning and execution with timelines; (iv) the validation{\dag}of this approach in real-world manufacturing scenarios.

Tables

Table 1. Table 4.1 : Defined temporal relations

the relation	is defined as
$A\mathrm{meets}B$	$A{\mathrm{end}\_\mathrm{before}\_\mathrm{start}}_{[0,0]}B$
$A{\mathrm{before}}_{[lb,ub]}B$	$A{\mathrm{end}\_\mathrm{before}\_\mathrm{start}}_{[lb,ub]}B$
$A{\mathrm{overlaps}}_{[l{b}_1,u{b}_1][l{b}_2,u{b}_2]}B$	$A{\mathrm{start}\_\mathrm{before}\_\mathrm{start}}_{[l{b}_1,u{b}_1]}B\wedge$
	$A{\mathrm{end}\_\mathrm{before}\_\mathrm{end}}_{[l{b}_2,u{b}_2]}B\wedge$
	$B{\mathrm{start}\_\mathrm{before}\_\mathrm{end}}_{[0,\infty ]}A$
$A\mathrm{equals}B$	$A{\mathrm{start}\_\mathrm{before}\_\mathrm{start}}_{[0,0]}B\wedge$
	$A{\mathrm{end}\_\mathrm{before}\_\mathrm{end}}_{[0,0]}B$
$A{\mathrm{contains}}_{[l{b}_1,u{b}_1][l{b}_2,u{b}_2]}B$	$A{\mathrm{start}\_\mathrm{before}\_\mathrm{start}}_{[l{b}_1,u{b}_1]}B\wedge$
	$B{\mathrm{end}\_\mathrm{before}\_\mathrm{end}}_{[l{b}_2,u{b}_2]}A$
$A{\mathrm{starts}}_{[lb,ub]}B$	$A{\mathrm{start}\_\mathrm{before}\_\mathrm{start}}_{[0,0]}B\wedge$
	$A{\mathrm{end}\_\mathrm{before}\_\mathrm{end}}_{[lb,ub]}B$
$A{\mathrm{finishes}}_{[lb,ub]}B$	$A{\mathrm{start}\_\mathrm{before}\_\mathrm{start}}_{[lb,ub]}B\wedge$
	$A{\mathrm{end}\_\mathrm{before}\_\mathrm{end}}_{[0,0]}B$
$A\mathrm{starts}\_\mathrm{at}t$	$A{\mathrm{starts}\_\mathrm{before}}_{[0,0]}t$
$A\mathrm{ends}\_\mathrm{at}t$	$A{\mathrm{ends}\_\mathrm{before}}_{[0,0]}t$

Equations

$$A(x_1,...,x_n,s{t}_A,e{t}_A,\delta )$$

$$C[A_i]:G(\vec{y})\to T(A_i,A_j,...,A_k)\wedge P(\overrightarrow{x_i},\overrightarrow{x_j},...,\overrightarrow{x_k})$$

$$R[A(\vec{x},st,e{t}_A,\delta )]=C_1[A]\wedge ...\wedge C_n[A]$$

$$hold(att(x_1,...):v,(t_1,t_2))$$

$$event(att(x_1,...):(v_1,v_2),t)$$

$$use(typ(r):q,(t_1,t_2))$$

$$consume(typ(r):q,t)$$

$$produce(typ(r):q,t)$$

$$x=\left(V,D,T\right)$$

$$x_i=\left(v_j,[s_i,s_i^{\prime }],[e_i,e_i^{\prime }]\right)$$

$$x^i=(v,[e,e^{\prime }],[d,d^{\prime }],\gamma (v))$$

$$\begin{array}[]{cc}x^{1}=(v_{1},[e_{1},e^{\prime}_{1}],[d_{1},d^{\prime}_{1}],\gamma(v_{1})),\\ \dots,\\ x^{k}=(v_{k},[e_{k},e^{\prime}_{k}],[d_{k},d^{\prime}_{k}],\gamma(v_{k})),\end{array}$$

$$\begin{array}[]{l}inst^{1}=(Stowed,[20,28],[20,30],c)\\ inst^{2}=(Unstowing,[23,31],[3,3],c)\\ inst^{3}=(Unstowed,[50,55],[19,32],c)\end{array}$$

$$\begin{array}[]{l}inst^{1}=(Stowed,[25,25],[20,30],c)\\ inst^{2}=(Unstowing,[28,28],[3,3],c)\\ inst^{3}=(Unstowed,[50,50],[19,32],c)\end{array}$$

$$x^i=(v,[t,t],[d,d^{\prime }],\gamma (v))$$

$$\begin{array}[]{ll}\mathit{FTL}_{inst}=&inst^{1}=(Stowed,[20,28],[20,30],c)\\ &inst^{2}=(Unstowing,[23,31],[3,3],c)\\ &inst^{3}=(Unstowed,[50,55],[19,32],c)\\ \end{array}$$

$$\begin{array}[]{ll}\mathit{STL}_{inst}=&inst^{1}=(Stowed,25,[20,30],c)\\ &inst^{2}=(Unstowing,28,[3,3],c)\\ &inst^{3}=(Unstowed,51,[19,32],c)\\ \end{array}$$

$$\begin{array}[]{l}cm^{5}=(Idle,100,[1,43],c)\\ cm^{6}=(SendData,123,[11,32],u)\end{array}$$

$$\begin{array}[]{l}win^{1}=(NotAvailable,60,[60,80],u)\\ win^{2}=(Available,130,[50,90],u)\end{array}$$

$$\exists\,a_{1}[x_{1}=v_{1}]\dots\ a_{n}[x_{n}=v_{n}]\,.\,\mbox{$\cal C$}$$

$$a_0[x_0=v_0]\to {\mathcal{E}}_1\vee \cdots \vee {\mathcal{E}}_k$$

$$\top \to {\mathcal{E}}_1\vee \cdots \vee {\mathcal{E}}_k$$

$$\exists\ a_{1}[x_{1}=v_{1}]\dots a_{n}[x_{n}=v_{n}]\,.\,\mbox{$\cal C$}$$

$$S=a_0[x_0=v_0]\to {\mathcal{E}}_1\vee \cdots \vee {\mathcal{E}}_k$$

$$\begin{array}[]{ll}\mathit{STL}_{cm}=&\dots\\ &cm^{i-1}=(Idle,110,[50,80],c),\\ &cm^{i}=(SendData,130,[11,32],u)\end{array}$$

$$\begin{array}[]{ll}\mathit{STL}_{win}=&\dots\\ &win^{j-1}=(NotAvailable,80,[1,100],u),\\ &win^{j}=(Available,170,[60,100],u)\\[5.69046pt] \mathit{STL}_{nav}=&\dots\\ &nav^{k-1}=(Moving,95,[14,32],u),\\ &nav^{k}=(At,185,[34,95],c)\end{array}$$

$$\begin{array}[]{l l}\mathit{STL}_{r}=&r^{1}=(Idle,23,[11,200],c),\\ &r^{2}=(TakeSample,55,[1,70],c),\\ &r^{3}=(Idle,200,[23,178],c)\end{array}$$

$$\begin{array}[]{l l}\mathit{STL}_{inst}=&inst^{1}=(Stowed,28,[9,45],c),\\ &inst^{2}=(Unstowing,31,[3,3],c),\\ &inst^{3}=(Unstowed,32,[1,30],c),\\ &inst^{4}=(Placing,35,[3,7],c),\\ &inst^{5}=(Sampling,42,[7,18],u),\\ &inst^{6}=(Unstowed,200,[1,200],c)\end{array}$$

$$\begin{array}[]{l l}\mathit{STL}_{nav}=&nav^{1}=(Home,5,[5,5],c),\\ &nav^{2}=(Moving,27,[19,37],u),\\ &nav^{3}=(At,200,[1,200],c)\end{array}$$

$$\begin{array}[]{l l}\mathit{STL}_{cm}=&cm^{1}=(Idle,65,[50,80],c),\\ &cm^{2}=(SendData,83,[11,32],u),\\ &cm^{3}=(Idle,200,[1,200],c)\end{array}$$

Full text

1. 1 Introduction

2. 2 Planning in Artificial Intelligence

3. 2.1 Classical Planning

4. 2.1.1 STRIPS

5. 2.1.2 PDDL

6. 2.1.3 HTN

7. 2.2 Planning in the Real-World

8. 2.3 Temporal Planning

9. 2.3.1 PDDL2.1

10. 2.3.2 Hybrid Planning approaches

11. 3 Timeline-based Planning in a Nutshell

12. 3.1 EUROPA

13. 3.2 IxTeT

14. 3.3 APSI-TRF

15. 3.4 Temporal Formalisms

16. 3.4.1 The Simple Temporal Problem

17. 3.4.2 The Simple Temporal Problem with Uncertainty

18. 4 Flexible Timeline-based Planning with Uncertainty

19. 4.1 A Running Example: The Rover Domain

20. 4.2 Domain Specification

21. 4.2.1 State Variables

22. 4.2.2 Restricting the Behavior of State Variables

23. 4.2.3 Planning Domains

24. 4.3 Flexible Plans

25. 4.4 Problem Specification

26. 4.4.1 Planning Goals

27. 4.4.2 Solution Plans

28. 5 The Extensible Planning and Scheduling Library

29. 5.1 The Modeling Language

30. 5.1.1 The Domain Description Language

31. 5.1.2 The Problem Description Language

32. 5.2 Architectural Overview

33. 5.2.1 Representation Framework

34. 5.2.2 Problem Solving

35. 5.2.3 The General Solving Procedure

36. 5.3 Looking for Pseudo-controllable Plans

37. 5.4 Hierarchical Planning with Timelines

38. 5.4.1 Hierarchical Modeling Approach

39. 5.4.2 Building the Dependency Graph

40. 6 Planning and Execution with Timelines under Uncertainty

41. 6.1 Model-based Control Architectures

42. 6.2 Extending the EPSL framework with Execution

43. 6.2.1 The Execution Process

44. 6.2.2 Managing the Execution Dependency Graph

45. 6.2.3 Handling Uncertainty During Execution

46. 6.2.4 The Importance of Being (Temporally) Robust

47. 6.3 Human-Robot Collaboration: a Case Study

48. 6.3.1 The FourByThree Research Project

49. 6.3.2 Assembly/Disassembly Operation

50. 6.4 Dynamic Task Planning in FourByThree

51. 6.4.1 Task Planning Model for Assembly/Disassembly

52. 6.4.2 Feasibility Check of the Task Planning Model

53. 6.4.3 The Dynamic Task Planning Module in Action

54. 7 Knowledge-based Control Loop for Flexible Controllers

55. 7.1 Flexible Plan-based Control Architectures

56. 7.1.1 The Manufacturing Case Study

57. 7.1.2 The Use of Ontologies in Manufacturing

58. 7.2 Knowledge and Plan-based Control in a Loop

59. 7.2.1 The Knowledge-based Control Loop at Runtime

60. 7.3 Modeling Knowledge with Ontology and Contexts

61. 7.3.1 The DOLCE Ontology

62. 7.3.2 Ontological interpretation of Agents and their Environment

63. 7.3.3 Ontology and Engineering of Functions

64. 7.3.4 Context-based Characterization

65. 7.3.5 Applying Ontology and Contexts to the Case Study

66. 7.4 The Knowledge-Base Life Cycle

67. 7.4.1 The Low-Level Reasoning Step

68. 7.4.2 The High-Level Reasoning Step

69. 7.5 Generating the (Timeline-based) Control Model

70. 7.5.1 Building State Variables from Contexts

71. 7.5.2 Building Decomposition Rules from Inference Trace

72. 7.5.3 The Resulting Timeline-based Control Model

73. 7.6 The Knowledge-Based Control Loop in Action

74. 8 Concluding Remarks

1. 2.1 The state space of the Vacuum World domain
2. 3.1 A graph representation of the STP problem described in Example 3.4.1
3. 4.1 State Variable specification for the Rover planning domain
4. 5.1 The layered architecture of the EPSL framework
5. 5.2 The structure of the plan database in the EPSL framework architecture
6. 5.3 The structure of a planner in the EPSL framework architecture
7. 5.4 Hierarchical modeling of timeline-based domains
8. 5.5 Extracting domain hierarchy from synchronizations’ constraints
9. 6.1 The EPSL architecture extended with executive capabilities
10. 6.2 The structure of the executive in the EPSL framework
11. 6.3 The structure and interactions of the executive control cycle
12. 6.4 Management of the received feedback signals during the control cycle
13. 6.5 Management of the dispatching step during the control cycle
14. 6.6 Different execution state transitions for: (a) controllable tokens; (b) partially-controllable tokens; (c) fully-uncontrollable tokens. State transitions tagged with "c" are controllable while transitions tagged with "u" are uncontrollable
15. 6.7 The overall ALFA pilot production process
16. 6.8 The manual procedure of the Assembly/Disassembly process of the ALFA pilot
17. 6.9 The hierarchy of the task planning domain
18. 6.10 Defining the workflow of work-cell operations
19. 6.11 Assigning tasks to the robot and the human
20. 6.12 Decomposition of the low-level tasks of the robot controller
21. 6.13 The Gantt chart representation of the plan for the ALFA pilot with respect to the earliest start time of the related tokens
22. 6.14 Deliberation time on different problems with different assignment policies
23. 6.15 Process-view of the dynamic task planning control module
24. 6.16 ROS-based simulation of the dynamic task planning system
25. 7.1 A picture of a Transportation Module (on the left) of the RTS and two possible configurations (on the right)
26. 7.2 The Knowledge-based Control Loop
27. 7.3 The DOLCE taxonomy of particulars
28. 7.4 The function ontological taxonomy and its rationale
29. 7.5 Extension of DOLCE ontology
30. 7.6 The general class axiom for the transportation module category
31. 7.7 The knowledge processing mechanism
32. 7.8 Raw data received from the Diagnosis Module
33. 7.9 Inferring collaborators of a TM
34. 7.10 Inferring primitive channels of a TM
35. 7.11 Inferring complex channels of a TM
36. 7.12 A (partial) view of the timeline-based model generated for a TM equipped with one cross-transfer unit only
37. 7.13 KB initial inference and planning domain generation
38. 7.14 KB inference and planning domain generation during KBCL reconfiguration phase
39. 7.15 Deliberation time with increasing number of goals and different TM configurations during KBCL setup phase

Chapter 1 Introduction

"Maybe the only significant difference between a really smart simulation and a human being was the noise they made when you punched them."

• Terry Pratchett, The Long Earth

Artificial Intelligence (AI) is the field of Computer Science that deals with the development of techniques that aim at endowing machines with some sort of intelligence. There are different research fields in AI that characterize intelligence in different ways and realize different types of intelligent machines accordingly. Broadly speaking, "the term artificial intelligence is applied when a machine mimics cognitive functions that humans associate with other human minds, such as learning and problem solving" as stated in [Russell and Norvig, 2003]. Automated Planning is one of the core fields of AI since its beginnings. Its research objective is to endow a machine (an artificial agent) with the capability of autonomously carry out complex tasks. From a practical point of view, this is a key enabling feature in application scenarios where direct human involvement is neither possible nor safe, e.g. space mission or deep sea exploration. Moreover, the recent and continuous improvements of robotic platforms with respect to reliability and efficiency represent a great opportunity for deploying AI-based techniques in even more common application scenarios (e.g. domestic care, manufacturing, rescue missions).

A planner is a general problem solver able to automatically synthesize a set of actions that allow an agent to achieve some objectives (e.g. explore and gather scientific data about an unknown environment or accomplish some complex task within the production process of a factory). A planning system usually relies on a model which represents a general description of the world. The model characterizes the environment the agent is supposed to operate in, and the agent capabilities in terms of the actions the agent can perform to interact with the environment. The classical approach to planning relies on a logical characterization of the model that focuses on the causal aspects of the problem to solve. States consist of sets of atoms asserting known facts and properties about the world, e.g. the position of a robot or an object in the environment. Actions encode transitions between states by specifying preconditions and effects. Preconditions specify a set of conditions that must be true (i.e. atoms) in order to apply the action in a particular state. Effects specify conditions that become true (i.e. positive effects) or false (i.e. negative effects) after the application of the action. For example, the action of moving an object from an initial location to a destination location can be applied to all states in which the object to move is located at initial location. The states resulting from the application of the action are all those states in which the object is located at the destination location. A planning goal usually consists of a logical formula representing the goal state to achieve, e.g. the state in which all objects of the domain are in a desired location. Thus, given an initial state containing a set of known facts about the world (e.g. the initial locations of the objects), the planning system must synthesize a set of actions needed to reach the goal state.

STRIPS [Fikes and Nilsson, 1971] is one of the first planning system introduced into the literature whose language inspired the PDDL [Mcdermott et al., 1998] which is the standard modeling language for planning. Many planning systems have been developed, e.g. SATplan [Kautz and Selman, 1992], FF [Hoffmann and Nebel, 2011] or LPG [Gerevini and Serina, 2002], that rely on the classical planning formalism and PDDL language. These planners have also shown relevant solving capabilities on toy problems during the International Planning Competitions. However, from a practical point of view, the classical planning formalism makes strong assumptions on the features of the problems to model. These assumptions limit the capabilities of classical planners to address real-world problems. For example, classical planning paradigms use an implicit representation of time which does not allow planners to deal with concurrency, temporal constraints or durative actions that are crucial in real-world scenarios. Consequently, several extensions to classical planning have been introduced into the literature in order to overcome these limitations and address more realistic problems. These extensions lead to the definition of several planning paradigms that relax different assumptions of the classical approach. Temporal planning represents the class of planning paradigms that introduce an explicit representation of time into the modeling language.

The timeline-based approach is a particular temporal planning paradigm introduced in early 90’s with HSTS [Muscettola, 1994] which has been successfully applied to solve many real-world problems (in space-like contexts mainly). This approach takes inspiration from classical control theory and is characterized by a more practical than logical view of planning. The timeline-based approach focuses on the temporal behavior of a system and the related features that must be controlled. Specifically, a complex system (e.g. an exploration rover) is modeled by identifying a set of relevant features that must be controlled over time within a known temporal horizon (e.g. the wheeled base of the robot or the communication facility). The control process consists in the synthesis of a set of temporal behaviors (i.e. the timelines) that describe how the modeled features evolve over time. The main advantage of planning with timelines consists in the representation approach, which allows the planner to deal with time and temporal constraints while building a plan. Namely, the timeline-based representation fosters a hybrid solving procedure by means of which it is possible to integrate planning and scheduling in a unified reasoning mechanism. In general, hybrid reasoning is essential to effectively address real world problems. Indeed, the key factor influencing the successful application of planning technologies to real-world problems is the capability of simultaneously dealing with different aspects of the problem like causality, time, resources, concurrency or uncertainty at solving time.

Despite the practical success of timeline-based approach, formal frameworks characterizing this formalism have been proposed only recently. There are several planning systems that have been introduced into the literature, e.g. EUROPA [Barreiro et al., 2012], IxTeT [Ghallab and Laruelle, 1994], APSI-TRF [Fratini et al., 2011], each of which applies its own interpretation of timeline-based planning. Moreover, developed Planning and Scheduling (P&S) applications are strictly connected to the specific context they have been designed for. Thus, existing timeline-based applications are hard to adapt to different problems. In general, there is a lack of methodology in modeling and solving timeline-based problems. Given the elements that compose a particular domain, it is not easy to design a suited model in order to ralize effettive P&S solutions. In addition, different systems apply different solving approaches and generate plans with different features. Thus, it is not simple to compare different timeline-based systems and it is even more difficult to compare such systems with other existing approaches.

Contribution

The objective of this work is to investigate timeline-based planning by taking into account several aspects ranging from the semantics of the main planning concepts to the modeling and solving approach. Thus the contribution of the work involves (i) the proposal of a formal characterization of the timeline-based approach which takes into account also temporal uncertainty, (ii) the proposal of a hierarchical modeling and solving approach, (iii) the development of a general-purpose framework for planning and execution with timelines (EPSL - Extensible Planning and Scheduling Library), which complies with the proposed formalization and implements the proposed hierarchical solving procedure and lastly (iv) the validation of EPSL and the envisaged approach to timeline-based planning in real-world manufacturing scenarios.

The proposed formalization defines a clear semantics of concepts like timelines, timeline-based plans and state variables, representing the basic building blocks of a planning domain. In particular, the formalization takes into account the controllability properties in order to model the temporal uncertainty concerning the uncontrollable features of a domain. This is particularly relevant for real-world problems, where not all the features of the domain are controllable with respect to the point of view of the artificial agent. Namely, the environment has uncontrollable dynamics that may affect the behavior of the system to control and the outcome of its operations (e.g. the visibility of the ground station for the communication operations of a satellite). The timeline-based plans, generated according to the proposed formalization, contain information about the uncertainty of the domain that can be analyzed to characterize the robustness of the plan with respect to its execution. There are several works in this field [Vidal and Fargier, 1999, Morris et al., 2001, Cesta et al., 2010] aiming at analyzing the plan in order to understand if, given the possible evolutions of the uncontrollable features of the domain, it is possible to complete the execution of the plan. With respect to planning, it is important to leverage the controllability information about the domain during the solving process in order to generate plans with some desired controllability properties (if possible) and therefore, have some information regarding their executability.

Given an agent to control, the proposed modeling approach follows a hierarchical specification of the domain which is similar to HTN planing [Georgievski and Aiello, 2015]. Specifically the approach proposes a functional characterization of the agent at different levels of abstraction. Broadly speaking, a primitive level characterizes the functional behavior of the physical/logical elements composing the agent in terms of commands they can directly manage over time. Functional levels model complex functions/operations the agent could perform over time by leveraging its components. Namely, functional levels model complex activities (i.e. complex tasks) the agent can perform by combining the available commands (i.e. primitive tasks). Domain rules, like methods in HTN planning, describe the operational constraints that allow the agent to implement tasks. They specify hierarchical decomposition of complex tasks in sets of constraints between primitive tasks. The resulting hierarchical structure encodes specific knowledge about the domain that the planning system can leverage during solving. Specifically, this work introduces search heuristics that leverages the hierarchical structure of the planning domain to support the plan generation process.

The EPSL framework complies with both the formalization and hierarchical modeling/solving approaches presented. EPSL is the major result of this study. It realizes a uniform framework for planning and execution with timelines under uncertainty. From the planning point of view, EPSL implements a hierarchical solving approach which is capable of dealing with temporal uncertainty during plan generation. Specifically, the solving procedure leverages information about the temporal uncertainty of the planning domain in order to generate plans with some properties characterizing their robustness with respect to the execution in the real-world. From the execution point of view, EPSL executes the timeline of a plan by taking into account the controllability properties of the related values and adapting the plans to the unexpected behaviors of the environment. EPSL planning and execution capabilities have been successfully applied to real-world manufacturing scenarios showing the effectiveness of the proposed approach.

Outline

Chapter 2 provides a brief description of the background of Automated Planning in AI by describing the classical approaches to planning, the limit of these approaches in solving real-world problems and how they have been improved in order to address more realistic problems. Chapter 3 provides a more detailed overview of the timeline-based planning approach and the related state of the art prior to this study. In particular, this chapter describes some of the most relevant timeline-based systems introduced into the literature (EUROPA, IxTeT and APSI-TRF) together with a brief description of the temporal formalisms this kind of systems usually relies on. Chapter 4 enters into the details of the contribution of the study by describing the proposed formalization of the timeline-based approach and the related controllability problem. Chapter 5 describes EPSL its structure and the implemented hierarchical modeling and solving approach. Chapter 6 presents a relevant extension of EPSL that allows the framework to execute plans while managing temporal uncertainty. This chapter also describe the deployment of EPSL to an interesting real-world manufacturing scenario of Human-Robot Collaboration (HRC). In particular, HRC applications represent well-suited contexts to leverage the EPSL capabilities of dealing with temporal uncertainty at planning and execution time. Chapter 7 presents another interesting application of the EPSL framework and its integration with semantic technologies for realizing an extended plan-based control architecture. Specifically, the chapter presents a flexible control architecture, called KBCL (Knowledge-based Control Loop), which has been applied to a real-world scenario for controlling reconfigurable manufacturing systems. KBCL aims at realizing a flexible control process able to dynamically adapt the control model to the different situations that may affect the capabilities of the system. KBCL investigates the integration and the correlations of ontological analysis and knowledge processing with the timeline-based planning approach. Finally chapter 8 draws some conclusions by providing an assessment of the achieved results and illustrates some of the most relevant open points that must be addressed in the near future.

Chapter 2 Planning in Artificial Intelligence

Planning is one of the most relevant research field of AI since its beginnings. The objective of a planning system is to automatically solve a problem by synthesizing a set of operations (i.e. a plan) needed to reach a desired goal (i.e. a desired state or configuration). There are many practical field like robotics or manufacturing where planning technologies have provided a significant contribution. Let us consider, for example, planetary exploration rovers that must operate in a context where direct human control is not possible. In such a context, planning technologies provide the rover with the autonomy needed for navigating an unknown environment and gathering scientific data to communicate.

There are different ways to describe the fundamental elements of a planning system. Such differences have lead to different planning paradigms ranging from those addressing fully observable, deterministic, static and discrete environments, to those that deal with partially observable stochastic environments. This chapter does not aim at presenting a complete background on all the planning technologies and systems that have been introduced into the literature. Thus, after a brief overview of some classical approaches to planning in section 2.1, section 2.2 explains the limits of these planning paradigms and the improvements needed to address real-world problems. Finally, section 2.3 focuses on a particular class of planning paradigms (i.e. Temporal Planning) which extends the classical approach by introducing an explicit representation of time.

2.1 Classical Planning

Broadly speaking, a planning system is a general problem solver whose aim is to synthesize a set of operations that, given an initial state, allow the system to reach a desired goal state. The reasoning process relies on a model which represents a general description of the problem to solve. The model provides a representation of the environment in terms of the possible states of the world and the actions the system can perform to interact with the environment. Thus, a planning process starts from an initial state and iteratively moves to other states by applying the available actions until a desired goal state is reached.

An example of a simple planning problem is represented by the Vacuum World problem described in [Russell and Norvig, 2003]. The problem consists of a set of rooms that can be either clean or dirty, and a vacuum cleaner which can move between (adjacent) rooms and clean the room the vacuum is located in. In this regard, a state of the world describes the set of rooms that compose the environment, their connections (i.e. whether two rooms are adjacent or not), their states (i.e. whether the rooms are clean or not), and the current room of the vacuum cleaner. The goal state is the state of the world where all rooms are clean. The initial state describes the status of all the rooms and the particular room the vacuum cleaner is initially located in.

Figure 2.1 shows the state space for the Vacuum World problem with two adjacent rooms. The state space can be seen as a directed graph where the possible states of the world are the nodes and actions are the (directed) edges connecting two states of the world. Let us consider, for example, the state in Figure 2.1 where both the rooms are dirty and the cleaner is located in the room on the left. The execution of action R (i.e. move right) leads to the state where both the rooms are dirty and the cleaner is located in the room on the right. Similarly, the execution of action S (i.e. clean/suck) leads to the state where the cleaner has not changed its position, the room on the left is clean and the room on the right is dirty.

Given a state space like the one depicted in Figure 2.1 and a known initial state, the planning process must find a sequence of actions needed to reach the state where both rooms are clean. The Vacuum World problem described above is very simple because states are fully observable (e.g. it is always possible to know whether a room is clean or dirty), actions are deterministic (i.e. there is not uncertainty about the effects of actions) and the search space is small. Thus, the planning process must simply find a path on the graph (i.e. the search space) connecting the initial state with the goal state. However, planning problems are not always fully observable or deterministic and typically entail huge search spaces that cannot be explicitly represented. A more compact and expressive representation/description of planning problems is needed and therefore several modeling languages and planning paradigms have been introduced into the literature.

2.1.1 STRIPS

STRIPS (STanford Research Institute Problem Solver) [Fikes and Nilsson, 1971] is one of the first automated planner and language used in AI. The STRIPS modeling language has represented the basic formalism for many planning paradigms that have been introduced successively. The formalism relies on the first-order predicate calculus to represent the space of world models the planning system must search in order to find a particular world model (i.e. a state), where a desired goal formula is achieved. A world model consists of a set of clauses, i.e. formulas of first-order predicate logic that describe a particular situation concerning the environment and the agent. For example, considering a robotic planning problem the related world models will contain a set of formulas concerning the position of the robot and all objects of the environment. Operators are particular transition functions that allow the planning system to move from one world model to others. It is supposed that for each world model there exists at least one operator which could be applied to "transform" the related world model into another. Thus the resulting problem solver must find the appropriate composition of operators that transform an initial world model to a "final" world model which satisfies a goal condition (i.e. a particular logical formula).

The problem space of a STRIPS problem is defined by the initial world model, the set of available operators and the goal states. Operators are grouped by schema which models a set of instances of applicable operators. Let us consider for example the operator goto, used for moving a robot between two points on a floor. In such a case, there is a distinct operator for each pair of points of the floor. Therefore it is more convenient to group all these possible instances into an operator schema goto(m, n) parametrized by the initial and final positions (m and n respectively). Specifically, an operator schema describes the effects and the conditions under which the operator is applicable. Effects specify the list of formulas that must be added to the model (the add list) and a list of formulas that must be removed (the delete list). Let us consider the example described in [Fikes and Nilsson, 1971] concerning a operator push(k, m, n) which models the action of pushing an object k from m to n. Such an operation can be modeled by the code below where ATR(m) is a predicate stating that the robot is at location m, and AT(k, m) is a predicate stating that the object k is at location m.

{highlights} push(k, m,n) precondition: ATR(m) $\land$ AT(k, m) delete list: ATR(m), AT(k, m) add list: ATR(n), AT(k, n)

2.1.2 PDDL

The Problem Domain Description Language (PDDL) is an action-based language introduced in [Mcdermott et al., 1998] for the AIPS-98 planning competition. PDDL relies on the STRIPS formalism and aims at defining a standard syntax for expressing planning domains. An early design decision was to separate the description of parametrized actions of the domain from the description of the objects, initial conditions and goals that characterize problem instances. Thus the domain description defines the general rules and behaviors that characterize as specific application scenario/context. Given a domain description, a problem description instantiates a planning problem in terms of specific type and number of objects, initial conditions and goals. In this way, a particular domain description can be used to define many different problem descriptions. PDDL defines parametrized actions by using variables denoting elements of a particular problem instance. Indeed, variables are instantiated to objects of the specific problem description when actions are grounded for applications. Preconditions and effects of actions are logical propositions constructed from predicates, arguments (i.e. objects from a problem instance) and logical connectivities. Moreover, PDDL extends the expressive power of STRIPS formalism by including the ability to descirbe structured object types, specify types for action parameters, specify actions with negative preconditions and conditional effects, as well as introduce the use of quantification in expressing both pre- and post- conditions. The code below shows an example of a simple PDDL action which allows a rover to move between two locations.

{highlights} (: move :parameters (?r - rover ?from ?to - location) :precondition (and (at ?r ?from) (path ?from ?to)) :effect (and (not (at ?r ?from)) (at ?r ?to)) )

The action $move$ has one parameter denoting the particular rover which is moving, and two other parameters denoting the specific locations the rover moves from and to. Action preconditions specify the conditions that must hold to apply actions. An instance of the action $move$ can be applied if the rover, the action refers to, is at the starting location (i.e. the location denoted by variable $?from$) and there exists a path connecting the starting location with the destination (i.e. the location denoted by variable $?to$). Action effects specify the state resulting from the application of the action. Thus, once the action has been applied, the rover denoted by variable $?r$, is no longer at location $?from$ (negative effect) but is at location denoted by the variable $?to$. Note that no temporal information is associated with action descriptions. Therefore, effects of actions become valid (i.e. true) as soon as actions are applied. Namely, actions in PDDL are instantaneous and there is not an explicit representation of time.

2.1.3 HTN

Hierarchical Task Network (HTN) planning [Georgievski and Aiello, 2015] is a particular paradigm which relies on the PDDL-based formalism. Like PDDL, atoms represent states of the world and actions represent deterministic state transitions. However the objective of HTN planners like SHOP2 [Nau et al., 2003, Nau et al., 1999] or O-Plan [Currie and Tate, 1991] is to generate a sequence of actions that perform some tasks. A task represents an activity to perform which can be either primitive or compound. Primitive tasks are accomplished by planning operators that, like PDDL operators/actions, describe transitions between states of the world. Compound tasks represent complex activity that cannot be directly "executed" and need to be further decomposed into a set of "smaller" tasks. In HTN planning, the objective is to synthesize a set of actions (i.e. primitive tasks) realizing a complex activity (i.e. a compound task) rather than reaching a desired goal state like classical planners. Thus, HTN domain description consists of a set of operators that describe the primitive tasks and a set of methods that specify how to decompose complex tasks into subtasks. Methods specify the hierarchical task decomposition HTN planners uses to recursively decompose tasks into a set of subtasks. Methods decompose tasks until primitive tasks are found and no further decomposition is needed. The resulting decomposition tree encode domain specific knowledge describing the standard operating procedures to use in order to perform tasks. Such a knowledge supports and guides the solving process of HTN planners. Although HTN solving procedure is general and domain independent, method specification is domain-dependent and characterizes the specific procedure to follow in order to realize complex tasks in the considered domain.

{highlights} (:method ; head (transport-person ?p ?c2) ; precondition (and (at ?p ?c1) (aircraft ?a) (at ?a ?c3) (different ?c1 ?c3)) ; subtasks (:ordered (move-aircraft ?a ?c1) (board ?p ?a ?c1) (move-aircraft ?a ?c2) (debark ?p ?a ?c2)) )

The block of code above shows an simple example of a SHOP2 method defined in [Nau et al., 2003], for a simplified versione of the ZenoTravel domain of the AIPS-2002 Planning Competition. The method describes how to transport a person $?p$ by aircraft from a location $?c1$ to another location $?c2$ in the case that the aircraft is not located at $?c1$. The $ordered$ keyword concerns task decomposition and specifies the order the planner must follow to expand subtasks. Thus, first the aircraft moves to location $?c1$, then the aircraft boards the person $?p$, then the aircraft moves to location $?c2$ and finally the aircraft debarks the person $?p$.

2.2 Planning in the Real-World

The modeling features of classical planning approaches rely on a set of assumptions that make strong simplifications of the problems to address with respect to real-world scenarios. Indeed, classical planning mainly deals with static, fully observable and deterministic domains. It means that given any state of the environment and a particular action, it is possible to know exactly which is the next state of the system. Such an assumption does not hold in real-world contexts where the environment may be partially observable and something may be either unknown or unpredictable. In such a case the planning system should be able to handle the uncertainty of the domain and find a sequence of actions that still reach the goal state.

Let us consider again for example, the Vacuum World domain of Figure 2.1, where the environment described is fully observable. At any state it is possible to know where the vacuum cleaner is located or it is possible to know exactly whether a room is clean or dirty. Similarly, the actions of the vacuum cleaner are deterministic and therefore, the state resulting from the application of an action is known. Let us consider, for example, the state where both rooms are dirty and the vacuum cleaner is located in the left room. If the the Suck operation is applied to this state, then the (only) successor state is the one with left room clean, the right room dirty and the vacuum cleaner still located in the left room. Such a simple problem can be made more "realistic" and more challenging if one or more assumptions are removed. Let us suppose to remove the assumption about the full observability of the environment and that it is not possible to know whether the rooms are clean or dirty. In such a case, it is necessary to find a sequence of actions that, independently from the actual state of the rooms, allows the system to reach a state where certainly both rooms are clean. Moreover, classical planning approaches have an implicit representation of time. Actions are supposed to be instantaneous, which means that the effects of an action become true as soon as the action is applied. States and/or goals are not supposed to have a temporal extension such that they hold only for a limited temporal interval, or that they must be achieved within a known temporal bound. Again this is a significant simplification in real-world contexts where time, temporal constraints (e.g. deadlines for goal achievement) and concurrency (e.g. a limit on the number of jobs that a machine can perform simultaneously) represent strong requirements that must be satisfied by plans.

There are several PDDL-based planning systems e.g. SATplan [Kautz and Selman, 1992], FF [Hoffmann and Nebel, 2011], LPG [Gerevini and Serina, 2002] or LAMA [Richter and Westphal, 2010], that have shown excellent solving capabilities during the International Planning Competitions. However, all the assumptions described above limit the expressivity of classical planning systems and their efficacy to address real-world problems. Consequently several planning approaches have been developed, with the intention of overcoming these limitations by removing one or more of the simplifying assumptions described above. In particular, this work focuses on Temporal Planning which represents the "class" of planning approaches capable of representing information and constraints that concern the temporal features of the domain. These kind of systems realizes problem solvers that make both planning and scheduling decisions during the solving process. Timeline-based planning belongs to this class of planning techniques and it will be further discussed in the next chapter. The following sections provide a brief description of the key modeling features of Temporal Planning, a brief description of PDDL2.1 [Fox and Long, 2003], the temporal extension of PDDL, and other hybrid approaches that present some common features with timeline-based planning like ANML [Smith et al., 2008], FAPE [Dvorák et al., 2014] and CHIMP [Stock et al., 2015].

2.3 Temporal Planning

The primary distinct characteristic of temporal planning paradigms is that they synthesize plans by combining causal reasoning with reasoning about time and resources. They overcome the traditional division between planning and scheduling technologies. In this context, planning is intended as the generation of a system behaviour that satisfies certain desired conditions over a prefixed temporal horizon. Therefore, planning is not only the process of deciding which actions to perform in order to satisfy some desired conditions, but also deciding when to execute these actions in order to obtain some desired behavior of the system. Indeed, temporal planning systems try to integrate planning and scheduling in a unified solving process.

2.3.1 PDDL2.1

PDDL2.1 [Fox and Long, 2003] has been designed to allow PDDL-based systems to model and solve more realistic domains by introducing the capability of dealing with time. There are several planning systems that rely on this language, e.g. OPTIC [Benton et al., 2012], COLIN [Coles et al., 2012] POPF [Coles et al., 2010], which also maintains backward compatibility with PDDL. Existing PDDL domains are valid PDDL2.1 domains and valid PDDL plans are valid PDDL2.1 plans. A relevant contribution of PDDL2.1 is the introduction of discretized durative actions with temporally annotated conditions and effects. Conditions and effects must be temporally annotated in order to specify when a particular proposition must hold. Specifically, a proposition (i.e. a condition or an effect) can hold at the start of the interval of the action (i.e. the time point at which the action is applied), at the end of the interval (i.e. the time point at which the effects of the action are asserted) or over the entire interval (i.e. invariant over the duration of the action). The annotation of an effect specifies whether the related effect of the action is instantaneous (i.e. the effect becomes true as soon as the action is applied) or delayed (i.e. the effect becomes true when the action finishes). The code below shows a simple example of a durative action for loading a truck from the Dock-Worker Robots domain described in [Ghallab et al., 2004].

{highlights} (:durative-action load-truck :parameters (?t - truck) (?l - location) (?o - cargo) (?c - crane) :duration (= ?duration 5) :condition (and (at start (at ?t ?l)) (at start (at ?o ?l)) (at start (empty ?c)) (over all (at ?t ?l)) (at end (holding ?c ?o))) :effect (and (at end (in ?o ?t)) (at start (holding ?c ?o)) (at start (not (at ?o ?l))) (at end (not (holding ?c ?o)))) )

Invariant conditions of a durative action hold over the entire duration of the action and are specified by means of the over all keyword (see the code above). It is worth observing that, the over all keyword excludes the start point and the end point of the action interval which is considered as an open temporal interval. If a particular preposition $p$ must hold at the start, at the end and also during the entire duration of the action, it must be specified with three temporal constraints, i.e. (at start p), (over all p) and (at end p).

2.3.2 Hybrid Planning approaches

There are other languages and planning frameworks that integrate causal and temporal reasoning without directly extending PDDL. An interesting planning language is the Action Notation Modeling Language (ANML) [Smith et al., 2008]. ANML has been introduced as an alternative to existing (temporal) planning languages like PDDL2.1, the IxTeT language or NDDL (the EUROPA planning language). Broadly speaking ANML represents an high-level language whose aim is to uniformly support generative and HTN planning models and provide a clear and well-defined semantics compatible with PDDL family of languages. ANML relies on a strong notation of action and state, provides constructs for expressing common forms of action conditions and effects, supports rich temporal constraints and uses a variable/value representation.

{highlights} action Navigate (location from, to) { duration := 5 ; [all] { arm == stowed ; position == from :-> to ; batterycharge :consumes 2.0 } }

The code above shows an example of an high-level navigation action for a rover expressed in ANML. The action has two location parameters of type from and to and a fixed duration (5 time units). The temporal qualifier [all] means that the related statements (i.e. the statements contained by the adjacent block of code) are valid all along the duration of the action. Specifically, the first statement specifies that the arm of the rover is stowed over the entire action. The second statement specifies that the position of the rover is from at the start of the action (a condition), the position is undefined during action execution, and the position is to at the end of the action (an effect). The last statement specifies the amount of energy consumed by the action.

The Flexible Acting and Planning Environment (FAPE) is a recently introduced planning framework [Dvorák et al., 2014] which extends HTN planning with temporal reasoning by implementing the ANML language. Another recent planner worth to be considered is CHIMP [Stock et al., 2015]. CHIMP relies on its own modeling language and extends HTN planning domain representation with temporal representation by leveraging the functionalities of the meta-csp [Mansouri and Pecora, 2014].

Chapter 3 Timeline-based Planning in a Nutshell

The timeline-based approach is a Temporal Planning paradigm introduced in early 90’s [Muscettola, 1994], which takes inspiration from classical control theory. The main distinct factor is the centrality of time in the representation formalism. Unlike classical approaches, timeline-based planning puts time to the center of the solving approach by dealing with concurrency, temporal constraints and flexible durations. Timeline-based planning realizes a sort of hybrid representation and reasoning framework which allows a solver to "easily" interleave planning and scheduling decisions. This hybrid view of planning is one of the key characteristic for successfully addressing real-world problems. Timeline-based solvers have been successfully applied in real-world contexts (especially in space-like contexts) [Muscettola et al., 1992], [Jonsson et al., 2000a], [Cesta et al., 2007].

The world model of a timeline-based application is characterized by a set of features that must be controlled over time in order to realize a complex behavior/task of a particular system to control. A complex system (e.g. a planetary exploration rover) is modeled by identifying a set of features that are relevant from the control perspective (e.g. the stereo camera or the communication facility). Each feature is modeled in terms of the values it may assume over time and their related temporal durations. The temporal evolution of a feature is represented as a timeline which consists of an ordered sequence of valued temporal intervals, usually called tokens. These tokens describe the behavior of the feature within a given temporal horizon. In addition to the description of the features, the model may also specify domain rules that allow to further constrain the temporal behaviors of the features through temporal constraints. Such rules are necessary to achieve high-level goals (e.g. take and communicate pictures of a target) by coordinating different features properly. For example, a rule may require that a particular value of a feature occurs during a known temporal interval or that a token of a timeline must always occur before a particular token of another timeline. Thus, a timeline-based plan consists of a set of timelines and that must satisfy all the temporal constraints of the domain in order to be valid.

In timeline-based planning, unlike classical planning, there is not a clear distinction between states and actions. A valued temporal interval may represent either an action or a state the related feature must perform or assume over a particular temporal interval. Similarly planning goals do not represent simply states or conditions that must be achieved. Rather, a planning goal may be either a value that a particular feature is supposed to assume during a certain temporal interval, or a complex task (e.g. take a picture of a target) that must be performed within a given time. The solving process acts on an initial set of partially specified timelines representing the initial known facts about the world. The process completes the behaviors of these timelines by iteratively adding values and temporal constraints according to desired requirements (including planning goals). Thus, timeline-based planners realize a behavior-based approach to planning, whose focus is on constraining the temporal evolutions of the system rather than synthesizing a sequence of actions that allow to achieve a desired goal state.

There are several timeline-based systems that have been introduced into literature and successfully applied to real-world problems (especially in space-like contexts). EUROPA [Barreiro et al., 2012] developed by NASA, IxTeT [Ghallab and Laruelle, 1994] developed at LAAS-CNRS, and APSI-TRF [Fratini et al., 2011] developed for Esa, represent some of the most known existing frameworks in this field. The next sections provide a brief description of the most relevant features of these timeline-based planning frameworks.

3.1 EUROPA

The EUROPA  framework [Barreiro et al., 2012] relies on Constraint-based Temporal Planning (CBTP) [Frank and Jonsson, 2003] which is a Temporal Planning formalism successfully applied in many space application contexts by NASA. The CBTP modeling approach focuses on the temporal behaviour of the system we want to control and not just on the causality relationships. Therefore, a complex system (e.g. a planetary exploration rover) is modelled by identifying a set of relevant components that can independently evolve over time.

A component models a physical or logical feature of the system to be controlled by specifying a (finite) set of mutually exclusive activities the related feature may assume over time. An activity is an atomic formula of the form:

[TABLE]

where (i) $A$ is a predicate representing a particular condition of the world, (ii) $\vec{x}=\{x_{1},...,x_{n}\}$ are numerical or symbolic parameters of activities, (iii) $st_{A}$ and $et_{A}$ are temporal variables representing respectively the activity start and end times and (iv) $\delta=[\delta_{min},\delta_{max}]$ is an interval representing lower and upper bounds of activity’s duration.

Let $Act=\{A_{1}(\vec{x}_{1},\delta_{1}),...,A_{k})(\vec{x}_{k},\delta_{k})\}$ be the set of activities. In CBTP formalism a component $C_{i}$ is defined by a subset $Act_{i}=\{A_{i,1},...,A_{i,m}\}$ of $Act$, where activities $A_{i,j}$ represent possible states or actions of the component $C_{i}$. Components statically describe the possible temporal evolutions of the elements of the system. However, it is necessary to specify additional constraints in order to coordinate system’s element and guarantee the overall system safeness. CBTP considers two types of constraints: (i) Codesignation Constraints can impose equalities or inequalities between the parameters of activities; (ii) Temporal Constraints can model temporal constraints between activities by expressing either interval-based or point-based temporal predicates.

In general, CBTP models temporal constraints by extending the qualitative temporal interval relationships defined in [Allen, 1983] with quantitative information. Namely, the basic temporal relations between intervals are enriched with metric information, i.e. lower and upper bounds of the distances between temporal intervals. For example, the relation A before [10, 20] B states that the interval A must precede interval B not less than 10 time units and not more than 20 time units.

The causal and temporal constraints of the system are modeled by means of dedicated rules, called compatibilities, that specify interactions between a particular activity of a component and other activities that can either belong to the same component (internal compatibility) or to a different component (external compatibility). Compatibilities describe how a particular activity (the master) is related to other activities (the slave) by specifying a set of codesignation and/or temporal constraints that must be satisfied in order to build valid plans. Conditional compatibilities can be defined by means of guard constraints that "extend" the conditions under which the related compatibility can be applied. If the guard constraints of a compatibility are satisfied, then the related temporal and/or codesignation constraints can be applied. Given a set of activities $Act$, the compatibility for an activity $A_{i}(\vec{x_{i}},st{A_{i}},et_{A_{i}},\delta_{i})\in Act$ is defined as

[TABLE]

where (i) $G(\vec{y})\equiv g_{1}(\gamma_{1})\land...\land g_{m}(\gamma_{m}))$ is a conjunction of guard constraints, (ii) $T(A_{i},A_{j},...,A_{k})\equiv t_{1}(A_{i},A_{j})\land...\land t_{m}(A_{i},A_{k})$ is a conjunction of temporal constraints involving the activities $A_{i},A_{j},...,A_{k}$ and (iii) $P(\vec{x_{i}},\vec{x_{j}},...,\vec{x_{k}})\equiv p_{1}(\vec{x_{i}},\vec{x_{j}})\land...\land p_{n}(\vec{x_{i}},\vec{x_{k}})$ is a conjunction of codesignation constraints on variable $\vec{x_{i}}$ and variables in $\cup_{t=j}^{k}(\vec{x_{t}})$.

If a compatibility $C[A]$ specifies different constraints according to the different values a particular guard variable $g_{i}(\gamma_{i})$ may assume then, $C[A]$ represents a disjunctive compatibility. Given an activity $A(\vec{x},st,et,\delta)$, a configuration rule for $A$ is a conjunction of compatibilities and it is defined as

[TABLE]

The code below shows some compatibilities and configuration rules for a classical planning problem concerning the control of a planetary exploration rover.

{highlights} R[Unstow()] = { [meets Place(rock) $\land$ met_by Stowed()] }

R[Place($rock_b$)] = { [meets Use(inst, $rock_u$) $\land$ ($rock_u=rock_b$)] $\land$ [met_by Unstow()] $\land$ [contained_by MobilitySystem.At($rock_a$) $\land$ ($rock_a=rock_b$)] }

R[Use(inst, $rock_b$)] = { [$\gamma$ = 0 $\to$ meets Stow() $\gamma$ = 1 $\to$ meets Place($rock_p$ $\land$ ($rock_p\neq\ rock_b$)] $\land$ [met_by Place($rock_p$) $\land$ ($rock_p=rock_b$)] $\land$ [contained_by MobilitySystem.At($rock_a$) $\land$ ($rock_a=rock_b$)] }

Given the elements described above, a planning domain $D$ is defined by a set of components $C[D]=\{C_{1},...,C_{n}\}$, a set of activities $Act$ associated with each component and a set of evolution rules $R[D]=\{R[A_{1}],...,R[A_{m}]]\}$, the domain contains an evolution rule $R[A_{i}]$ for each activity $A_{i}\in Act$. The CBTP planning process aims at building a valid description of the temporal behaviors of the components within a temporal horizon where goal activities are scheduled at proper times. Thus, a planning problem consists of a planning horizon and an initial configuration which (partially) describes the behaviors of the components. A solution plan is represented by a temporal execution trace which specifies for each time point, the activity the components are supposed to execute.

3.2 IxTeT

IxTeT [Ghallab and Laruelle, 1994] is a temporal planning system which tries to integrate plan generation and scheduling into the same planning process. Some of the most important features of the IxTeT planning paradigm are: (i) an explicit representation of time with different types of metric constraints between time points; (ii) a powerful representation of the world through multi-valued attributes; (iii) the management of a large range of resource types (unsharable, sharable, consumable and producible); (iv) a task formalism allowing for the representation of complex macro-operators.

Properties of the world are described by a set of multi-valued state attributes and a set of resource attributes. A state attribute describes a particular feature of the domain as a key-mapping from some finite domains into a finite range (the value of the attribute). The code below shows an example of a domain feature modeling the possible location of a robot.

{highlights} attribute position(?robot) { ?robot $\in$ {robot1, robot2}; ?value $\in$ {RoomM, LabRoom1, LabRoom2}; }

A resource is defined as any substance, or set of objects whose cost or availability induces constraints on the actions that use them. So a resource can be either a single item with unit capacity (i.e. an unsharable resource) or an aggregate resource that can be shared simultaneously between different actions without violating its maximal capacity constraint.

{highlights} resource robots(?robot) { $?robot\in\ \{robot1,robot2\}$; capacity = 1; }

resource paper_on_robot() { capacity = 3; }

IxTeT defines different types of state attributes that can classified as: (i) rigid attributes (or atemporal) representing attributes whose value does not change over time (they express a structural relationship between their arguments); (ii) flexible attributes (or fluents) representing attributes whose value may change over time. Flexible attributes may be further classified in: (i) controllable attributes representing attribute whose change of values can be planned for (but they can even change independently from the planning system); (ii) contingent attributes representing attributes whose changes of values cannot be controlled.

Moreover, IxTeT relies on a reified logic formalism where fluents (i.e. flexible attributes) are temporally qualified by the hold and the event (temporal) predicates.

The hold predicate

[TABLE]

asserts the (temporal) persistence of the value of state attribute $att(x_{1},...)$ to $v$ for each $t:t_{1}\leq\ t<t_{2}$.

The event predicate

[TABLE]

asserts the instantaneous change of the value of att($x_{1}$, …) from $v_{1}$ to $v_{2}$ occurred at time t.

Similarly, resource availability profile and the resource usage by the different operators are described by means of use, consume and produce predicates.

The use predicate

[TABLE]

asserts the borrowing of an integer quantity $q$ of resource $typ(r)$ on the temporal interval $[t_{1},t_{2}]$.

The consume predicate

[TABLE]

asserts that a quantity $q$ of resource $typ(r)$ is consumed at time $t$.

The produce predicate

[TABLE]

asserts that a quantity $q$ of resource $typ(r)$ is produced at time $t$.

Temporal data representation and storage is managed by the time-map manager which relies on time-points as elementary primitives [Dechter et al., 1991]. Time is considered as a linearly ordered discrete set of instants. Time-points are seen as symbolic variables on which temporal constraints can be posted. IxTeT handles both symbolic constraints and numeric constraints expressed as a bounded interval $[I^{-},I^{+}]$ on the temporal distance between time points. The time-map manager is responsible for propagating constraints on time-points to check the global consistency of the network and to answer queries about the relative position of time-points.

Planning operators are represented by means of a hierarchy of tasks. A task is a temporal structure composed of: (i) a set of sub-tasks; (ii) a set of events describing the changes of the world the task causes; (iii) a set of assertions on state attributes to express the required conditions or the protection of some fact between two task events; (iv) a set of resource usage; (v) a set of temporal and instantiation constraints binding the different time-points and variables of the task. Tasks are deterministic operators without ramification effects that may also refer to other sub-tasks in order to express macro-operators. The code below shows an example of elementary task (i.e. a task without sub-tasks) for a robot in charge of the maintenance of a laboratory consisting in putting paper in a machine when it is out of paper:

{highlights} task feed_machine(?machine) (start, end) { variable ?room; place(?machine, ?room); hold(position(robot): ?room, (start, end)); event(machine_state: (out_of_paper, ok), end); consume(paper_on_robot(): 1, end); produce(trunk_size(): 1, end); (end - start) in [00:01:00, 00:02:00]; }

The initial plan is a particular task that describes a problem scenario by specifying: (i) the initial values for the set of instantiated state attributes (as a set of explained events); (ii) the expected changes on some contingent state attributes that will not be controlled by the planner (as a set of explained events); (ii) the expected availability profile of the resources (as a set of uses); (iv) the goals that must be achieved (usually, as a set of assertions).

3.3 APSI-TRF

APSI-TRF [Fratini et al., 2011] is a software framework developed for Esa whose aim is to support the design and development of P&S applications by leveraging the timeline-based approach. The APSI-TRF framework provides the designer with a ready-to-use software library for modeling planning and scheduling concepts in the form of timelines. Specifically, APSI-TRF relies on the same modeling assumptions of HSTS [Muscettola, 1994] and therefore, a complex system is modeled by identifying a set of relevant features to control over time. The APSI-TRF framework makes available the modeling language and the software functionalities needed to model timeline-based domains in shape of multi-valued state variables and synchronization rules.

Multi-valued state variables model the features of the domain by describing their allowed temporal behaviors. State variables model domain features by specifying the values, the related feature may assume over time, together with the allowed durations and transitions. Thus, a state variable $x$ is defined as the tuple

[TABLE]

where (i) $V$ is the set of values the variable $x$ can assume over time, (ii) $D:V\rightarrow\mathbb{R}\times\mathbb{R}$ is a duration function specifying for each value $v\in\ V$ the minimum and maximum duration and (iii) $T:V\rightarrow\ 2^{V}$ is a transition function specifying for each value $v\in\ V$ the set of allowed successors. State variables specify causal and temporal constraints of the single features of a planning problem. They specify local rules that allow a planning system to build the timelines of the features composing the domain. Given a state variable $x$, a timeline describes the sequence of values the variable assumes over time by specifying a sequence of valued temporal intervals called tokens. A token is defined as the tuple

[TABLE]

where $[s_{i},s^{\prime}_{i}]$ and $[e_{i},e^{\prime}_{i}]$ represent respectively the flexible start and end of the temporal interval during which the variable $x$ is supposed to assume the value $v_{j}\in\ V$.

Synchronization rules model causal and temporal constraints of a planning domain by specifying global relations between tokens of different variables. In general, whenever a particular token $x_{i}$ occurs on a timeline (i.e. the trigger) a synchronization rule specifies a set of different tokens (i.e. the targets) that must occur on other timelines and a set of temporal constraints between the trigger and targets of the rule that must hold in order to build valid temporal behaviors. Indeed, synchronization rules allow the planning system to further constrain the temporal behaviors of the state variables in order to build timelines that satisfy some desired planning goals. Temporal constraints of synchronization rules are modeled by extending the qualitative relationships of the Allen’s interval algebra [Allen, 1983], with quantitative information.

It is worth observing that APSI-TRF, unlike other timeline-based frameworks (e.g. the EUROPA and IxTeT frameworks mentioned above), is not a planner but a development library for designing planning applications. In this regard, OMPS [Fratini et al., 2008] represents a domain-dependent timeline-based solver which has been developed on-top of the APSI-TRF modeling functionalities and successfully applied in space-exploration scenario [Ceballos et al., 2011].

3.4 Temporal Formalisms

Temporal Planners rely on expressive temporal formalisms that allow these paradigms to deal with time and temporal constraints. Many timeline-based systems (including EUROPA, IxTeT and APSI-TRF) model temporal information about plans by extending the Allen’s interval algebra [Allen, 1983] in order to represent expressive temporal relations between the temporal elements of a plan.

Temporal information represents additional knowledge the planner must properly managed during the solving process. Thus, timeline-based planners must encapsulate temporal reasoning mechanisms that process temporal information in order to verify the (temporal) consistency of plans. Temporal reasoning mechanisms are usually implemented by leveraging the formalism of Temporal Networks [Dechter et al., 1991] which represents a flexible representation of temporal data as a network of time points (i.e. the nodes of the network) and distance constraints between time points (i.e. the edges of the network).

3.4.1 The Simple Temporal Problem

The Simple Temporal Problem (STP) is a well-known formalism introduced in [Dechter et al., 1991] which consists of a set of events that may occur over known temporal intervals and a set of requirement constraints that specify distance constraints on the temporal occurrences of pairs of events. The problem is to find a temporal allocation of the events satisfying all the requirement constraints (i.e. the distance constraint). Namely, temporal reasoning mechanisms try to find an assignment of events to time points such that all the temporal constraints are satisfied. This concept is known as temporal consistency and is central to STPs.

Below is the description of a simple scenario taken from [Dechter et al., 1991], representing an example of the type of problems and inference the STP formalism can support.

Example 3.4.1**.**

John goes to work by car (30-40 minutes). Fred goes to work in a carpool (40-50 minutes). Today John left home between 7:10 and 7:20, and Fred arrived at work between 8:00 and 8:10. We also know that John arrived at work about 10-20 minutes after Fred left home. We wish to answer queries such as: "Is the information in the story consistent?", "What are the possible times at which Fred left home?", and so on.

Figure 3.1 shows the STP problem of Example 3.4.1 in graph form. When STPs are shown as graphs where nodes represent events and edges represent requirement constraints, they are called Simple Temporal Networks (STNs). The node "0" of Figure 3.1 represents the temporal origin of the plan/problem, the "absolute" time 7:00 with respect to Example 3.4.1. Node "1" in Figure 3.1 is associated with the event representing the time at which John leaves home. The edge between node "0" and node "1" labeled "[10, 20]" models the fact "John left home between 7:10 and 7:20" as a distance constraint (i.e. a requirement constraint) between the two related events. A strong limitation of the STP formalism concerns disjunctive constraints. STP cannot represent and therefore, cannot reason about disjunctive temporal intervals on events. Considering Example 3.4.1, STP cannot model disjunctive assertions like "John goes to work either by car (30-40 minutes), or by bus (at least 60 minutes)". Disjunctive assertions represent alternative plans the planning process may generate accordingly by branching the search space.

From a planning perspective, the temporal part of the plan can be reduced to a STP by modeling the start and end times of the activities of the plan (e.g. the start and end times of the tokens of a timeline) as events of the STP. Temporal relations and/or duration constraints concerning the activities/actions of the plan can be easily translated in the STP as one or more requirement constraints involving events related to the start/end times of the activities of the plan. Thus a planning system can leverage the STP formalism to post ordering constraints between activities during the solving process and check the temporal assignment of the activities of the plan. Namely, a planning system can check the (temporal) consistency of a plan by verifying the existence of a valid schedule of all the activities.

3.4.2 The Simple Temporal Problem with Uncertainty

STP makes the assumption that all the events of the plan are controllable. It means that the planning system can decide the temporal allocation (i.e. the schedule) of all the events. However, this is not always possible in real-world settings. Indeed, the activities of a plan usually model real-world tasks/actions whose durations can be affected by exogenous factors and therefore, the planning system cannot decide the temporal allocation of these activities (e.g. the planner can decide the start time of the execution of an action but not the end time). Such activities are called uncontrollable. Thus, a more expressive temporal formalism is the Simple Temporal Problem with Uncertainty (STPU).

The STNU formalism takes into account both controllable and uncontrollable events. In this formalism an event is considered uncontrollable if it is the target of contingent constraints that are typically used to model uncontrollable durations of the activities/actions of the plan. The key point of STPUs is that temporal consistency is not sufficient to solve real-world problems. Temporal uncertainty introduces the additional problem of deciding how to schedule controllable events according to the observed/possible temporal occurrences of uncontrollable events, in order to complete the execution of the plan. Such a problem is called the controllability problem which has been fairly investigated in the literature [Vidal and Fargier, 1999, Morris et al., 2001]. Broadly speaking, three different types of controllability (weak, strong and dynamic controllability) have been defined according to the different assumptions made on the uncontrollable events of a plan. Planning systems may leverage the STPU formalism to generate plans with some desired properties concerning the controllability of generated plans (i.e. properties concerning the execution of the generated plans in the real-world).

Chapter 4 Flexible Timeline-based Planning with Uncertainty

Despite the practical success of timeline-based planning, formal frameworks characterizing this paradigm have been proposed only recently. There is a multitude of software frameworks that have been realized and introduced in the literature, each of which applies its own interpretation of timeline-based planning. In such a context, it is not easy to evaluate the modeling and solving capabilities of different timeline-based planning systems. It is not even easy to define benchmarking domains to compare timeline-based systems, or to open the assessment to other planning techniques.

This chapter describes a complete and comprehensive formal characterization of the timeline-based approach which has been introduced in [Cialdea Mayer et al., 2016]. The proposed formalization aims at defining a clear semantics of the main planning concepts by taking into account the features of the most known timeline-based planning frameworks. In addition, the formalization takes into account temporal uncertainty which is particularly relevant in real-world domains where not everything is controllable. Indeed, the execution process is not completely under the control of the executive system. Exogenous events can affect or even prevent the complete and successful execution of generated plans. Thus, the capability of representing and dealing with temporal uncertainty and controllability properties at both planning and execution time is crucial to deploy effective timeline-based applications in real-world scenarios.

4.1 A Running Example: The Rover Domain

In order to support the formal definitions given below, a simple case study will be used as a running example. The domain takes inspiration from a typical scenario of AI-based control for a single autonomous agent.

The Rover domain consists of an exploration rover which can autonomously navigate a (partially) unknown environment, take samples of some targets (e.g. rocks) and communicate scientific data to a satellite. An exploration rover is a complex system endowed with several devices that must be properly controlled in order to achieve the desired objectives. A navigation facility allows the rover to move and explore the environment. A dedicated instrument facility allows the rover to take samples of targets that must be analyzed. A communication facility allows the rover to send data acquired from sampled targets to a satellite whose orbit is known.

A mission goal requires the rover to move towards a desired target, take a sample of it and communicate gathered data when possible. All the features that compose the rover must be coordinated properly in order to realize the complex behavior needed to satisfy mission goals. Thus, a set of operative constraints must be satisfied. For instance, communication of data must be performed while the rover is still and during some known communication windows that represent temporal intervals during which the target satellite is visible. Another operative constraint requires that the instrument facility must be set in a safe position/configuration while the rover is moving.

4.2 Domain Specification

The timeline-based approach to planning pursues the general idea that planning and scheduling for controlling complex physical systems consist of the synthesis of desired temporal behaviors (or timelines). According to this paradigm, a domain is modeled as a set of features with an associated set of temporal functions on a finite set of values. The time-varying features are usually called multi-valued state variables [Muscettola, 1994]. Like in classical control theory, the evolution of the features is described by some causal laws and limited by domain constraints. These are modeled in a domain specification.

The task of a planner is to find a sequence of decisions that brings the timelines into a final desired set, satisfying the domain specification and special conditions called goals. Causal and temporal constraints specify which value transitions are allowed, the minimal and maximal duration of each valued interval and (so-called) synchronization constraints between different state variables. Moreover, a domain specification must take into account the temporal uncertainty of planning domains in order to model more realistic problems. In particular, two sources of uncertainty are considered.

On the one hand, the evolution of some components of the domain may be completely outside the control of the system. What the planner and the executive know about them is only what is specified in the underlying planning problem. On the other hand, some events may be partially controllable. In this case, the planner and the executive can decide when to start an activity, but they cannot fix the duration of the activity. According to this characterization, two types of state variables constitute a planning domain: the planned variables model the controllable or partially controllable features of a domain; the external variables model the uncontrollable features of a domain. Thus, the planning system or the executive must respectively make planning and execution decisions, without changing the behavior of external variables or making hypothesis on the actual duration of partially controllable features.

For the sake of generality, temporal instants and durations are taken from an infinite set of non-negative numbers $\mathbb{T}$, including [math]. For instance, $\mathbb{T}$ can be the set of natural numbers $\mathbb{N}$ (in a discrete time framework), as well as the non-negative real numbers $\mathbb{R}_{\geq 0}$. Sometimes, $\infty$ is given as an upper bound to allowed numeric values, with the meaning that $t<\infty$ for every $t\in\mathbb{T}$. The notation $\mathbb{T}^{\infty}$ will be used to denote $\mathbb{T}\cup\{\infty\}$, $\mathbb{T}_{>0}=\mathbb{T}-\{0\}$ and $\mathbb{T}^{\infty}{>0}=\mathbb{T}^{\infty}-\{0\}$. When dealing with temporal intervals, if $s,e\in\mathbb{T}$, the (closed) interval $[s,e]$ denotes the set of time points $\{t\mid s\leq t\leq e\}$.

4.2.1 State Variables

A state variable $x$ is characterized by four components: the set $V$ of values the variable may assume, a function $T$ mapping each value $v\in V$ to the set of values that are allowed to follow $v$, a function $\gamma$ tagging each value with information about its controllability, and a function $D$ which may set upper and lower bounds on the duration of each variable value.

Definition 4.2.2**.**

[State Variable]* A state variable $x$, where $x$ is a unique identifier, called the variable name, is a tuple $(V,T,\gamma,D)$, where:*

$V$, also denoted by $\operatorname{values}(x)$, is a non-empty set, whose elements are the state variable values. 2. 2.

$T:V\rightarrow 2^{V}$* is a total function, called the state variable value transition function.* 3. 3.

$\gamma:V\rightarrow\{c,u\}$* is a total function, called the controllability tagging function; $\gamma(v)$ is the controllability tag of the value $v$. If $\gamma(v)=c$, then $v$ is a controllable value, and if $\gamma(v)=u$, then $v$ is uncontrollable.* 4. 4.

$D:V\rightarrow\ \mathbb{T}\times\mathbb{T}^{\infty}$* is a total function such that $D(v)=(d_{min},d_{max})$ for some $d_{min}\geq 0$ and $d_{max}\geq d_{min}$, and if $\gamma(v)=u$, then $d_{min}>0$ and $d_{max}\neq\ \infty$; $D$ is called the state variable duration function.*

If $\gamma(v)=c$, then the planning or executive system can control the value $v$ and can decide the actual duration of related activities (e.g. the executive can decide when to start and end the execution of these activities). If $\gamma(v)=u$, then the planning or executive system cannot control the value $v$ and cannot decide the actual duration of related activities. The behaviors of these activities are under the control of the environment.

The intuition behind the duration function is that if $D(v)=(d_{min},d_{max})$, then the duration of each interval in which $x$ has the value $v$ is included between $d_{min}$ and $d_{max}$ inclusive, if $d_{max}\in\mathbb{T}$; it is not shorter than $d_{min}$ and has no upper bound, if $d_{max}=\infty$. In practice, existing systems, such as EUROPA [Barreiro et al., 2012] and APSI-TRF [Fratini et al., 2011], allow values to be represented by means of parametrized expressions In the present theoretical approach, values are taken to be completely instantiated in order to simplify the presentation. This amounts to describing sets and functions by enumeration and does not diminish the expressive power.

In what follows, it is assumed that, whenever a set of state variables $SV$ is considered, for every distinct pair $x$ and $y$ in $SV$, $x\neq y$. The set $SV$ is partitioned into two disjoints sets, $SV_{P}$, containing the planned state variables, and $SV_{E}$, the set of the external ones. Every value $v$ of an external state variable is uncontrollable, i.e., $\gamma(v)=u$. An external variable represents a component of the "external world" that is completely outside the system control: the planner cannot decide when to start or end its activities. What is known about an external variable is specified in the planning problem. On the contrary, a planned state variable represents a component of the system that is under the control of the executive. Nevertheless, controllable sub-systems may also have uncontrollable activities (i.e., activities whose starting times can be decided by the executive, but their durations and consequently their ending times, are not controllable). In other terms, the planner can decide when to start an uncontrollable activity of a planned variable (i.e. when the variable assumes an uncontrollable value), even if it cannot precisely predict how long it will last. In general, every time an activity (either controllable or not) is preceded by an uncontrollable one, the system cannot control its start time. Indeed, the start time of the activity is affected by the end of the previous activity, which is uncontrollable.

Example 4.2.3**.**

*In the considered running example, the timeline-based specification identifies five state variables, that will be called $r$ (for "rover"), $nv$ (for "navigator"), $inst$ (for "instrument"), $cm$ (for "communication") and $win$ (for "window") whose values, transitions and controllability properties are illustrated in Figure 4.1 (values with dotted borders represent uncontrollable values).

Therefore, the set of considered state variables is $SV=\{r,nv,inst,cm,win\}$ where $SV_{P}=\{r,\allowbreak nv,\allowbreak inst,\allowbreak cm\}$ are planned state variables, while $SV_{E}=\{win\}$ is an external one. For example, the state variable $inst$ models the instrument facility the rover uses to sample targets. The state variable can be defined by the typle $inst=(V_{inst},\allowbreak T_{inst},\allowbreak\gamma_{inst},\allowbreak D_{inst})$ where:*

• •

$V_{inst}=\{Stowed$, $Unstowed$, $Stowing$, $Unstowing$, $Placing$, $Placed$, $Sampling\}$;

• •

$T_{inst}$* is the value transition function such that*

• –

$T_{inst}(Stowed)=\{Unstowing\}$,

• –

$T_{inst}(Unstowed)=\{Stowing,\allowbreak Placing\}$,

• –

$T_{inst}(Stowing)=\{Stowed\}$,

• –

$T_{inst}(Unstowing)=\{Unstowed\}$,

• –

$T_{inst}(Placing)=\{Placed\}$,

• –

$T_{inst}(Placed)=\{Placing,\allowbreak Unstowed,\allowbreak Sampling\}$,

• –

$T_{Inst}(Sampling)=\{Placed\}$;

• •

$\gamma_{inst}$* is the controllability tagging function such that*

• –

$\gamma_{inst}(Stowed)=\gamma_{inst}(Unstowed)=\gamma_{inst}(Stowing)=\\ \gamma_{inst}(Unstowing)=\gamma_{inst}(Placing)=\gamma_{inst}(Placed)=c$,

• –

$\gamma_{inst}(Sampling)=u$;

• •

$D_{inst}$* is the value duration function such that*

• –

$D_{inst}(Stowed)=D_{inst}(Unstowed)=D_{inst}(Placed)=(1,\infty)$,

• –

$D_{inst}(Stowing)=D_{inst}(Unstowing)=(3,3)$,

• –

$D_{inst}(Placing)=(3,7)$,

• –

$D_{inst}(Sampling)=(7,18)$.

The state variable $win$, instead, is an external variable which models the availability of the communication channel during the satellite orbit. It can be defined by the tuple $win=(V_{win},\allowbreak T_{win},\allowbreak\gamma_{win},\allowbreak D_{win})$, where:

• •

$V_{win}=\{Available,\allowbreak NotAvailable\}$;

• •

$\gamma_{win}(Visible)=\gamma_{win}(NotAvailable)=u$;

• •

$T_{win}$* is the value transition function such that*

• –

$T_{win}(Available)=\{NotAvailable\}$**

• –

$T_{win}(NotAvailable)=\{Available\}$;

• •

$D_{win}$* is the duration function such that*

• –

$D_{win}(Available)=(60,100)$**

• –

$D_{win}(NotAvailable)=(1,100)$.

(Flexible) Timelines

A timeline represents the temporal evolution of a system component up to a given time. It consists of a sequence of valued intervals, called tokens, each of which represents a time slot in which the variable assumes a given value. A token represents a temporal interval which determines the instant the variable starts executing the related value and the time instant the variable ends executing that value.

However, planning with timelines takes into account time flexibility by allowing token durations to range within given bounds. It means that the start and end time instants of a token are replaced by temporal intervals. Thus, the notion of (flexible) timeline can be defined as follows:

Definition 4.2.4**.**

[Timeline]* If $x=(V,T,\gamma,D)$ is a state variable, a token for the variable $x$ has the form:*

[TABLE]

where $x^{i}$, for $i\in\mathbb{N}$, is the token name, $v\in V$, $e,e^{\prime},d,d^{\prime}\in\mathbb{T}$, $e\leq e^{\prime}$ and $d_{min}\leq d\leq d^{\prime}\leq d_{max}$, for $D(v)=(d_{min},d_{max})$. The value $\gamma(v)$ is called the token controllability tag; if $\gamma(v)=c$, then the token is controllable; if $\gamma(v)=u$, then the token is uncontrollable. A timeline $\mathit{FTL}_{x}$ for the state variable $x=(V,T,\gamma,D)$ is a finite sequence of tokens for $x$, of the form:

[TABLE]

where for all $i=1\dots k-1$, $v_{i+1}\in T(v_{i})$ and $e^{\prime}_{i}\leq e_{i+1}$. The interval $[e_{k},e^{\prime}_{k}]$ in the last token is called the horizon of the timeline and the number $k$ of tokens making up $\mathit{FTL}_{x}$ is its length. If $x^{i}=(v,[e,e^{\prime}],[d,d^{\prime}],\gamma(v))$ is a token in the timeline $\mathit{FTL}_{x}$, then:

• •

$\operatorname{val}(x^{i})=v$;

• •

$\operatorname{end\_time}(x^{i})=[e,e^{\prime}]$;

• •

$\operatorname{start\_time}(x^{0})=[0,0]$* and $\operatorname{start\_time}(x^{i+1})=\operatorname{end\_time}(x^{i})$;*

• •

$\operatorname{duration}(x^{i})=[d,d^{\prime}]$;

• •

with an abuse of notation, $\gamma(x^{i})$ denotes the token controllability tag $\gamma(\operatorname{val}(x^{i}))$.

Intuitively, a token $x^{i}$ of the above form represents the set of valued intervals starting at some $s\in\operatorname{start\_time}(x^{i})$, ending at some $e\in\operatorname{end\_time}(x^{i})$ and whose durations are in the range $\operatorname{duration}(x^{i})$. The horizon of the timeline is the end time of its last token.

Example 4.2.5**.**

Let us consider the timeline $\mathit{FTL}_{inst}$ for the state variable $inst$, in the Rover domain, made of the following sequence of tokens:

[TABLE]

*The horizon of $\mathit{FTL}_{inst}$ is $[50,55]$.

An example of non-flexible timeline for the same state variable $inst$ is made of the following sequence of tokens:*

[TABLE]

and its horizon is $[50,50]$.

It is worth pointing out that often in the literature (e.g., [Fratini et al., 2008]), a flexible token contains also a start interval. However, once a token $x^{i}$ is embedded in a timeline, the time interval to which its start point belongs ($\operatorname{start\_time}(x^{i})$) can be easily computed as shown in the definition above. Thus, including it as part of the token itself is redundant.

On the contrary, duration restrictions alone would be inadequate to precisely identify when the valued intervals represented by a given token must begin and end. As a matter of fact, duration and end time bounds interact when determining which legal values a token end time may assume. Let us assume, for instance, that the duration of a given token $x^{i}$ is $[20,30]$ and that one may compute, from the durations of the previous tokens, that its start time is $[40,50]$. One can then infer that the end points of the valued intervals it represents are necessarily in the range $[60,80]=[40+20,50+30]$. However, it may be the case that a stricter end time is required, for instance $[65,75]$. In this case, starting $x^{i}$ at $50$ and ending it at $80$, though respecting the duration bounds, would not be a legal value to "execute" the token, since $80\not\in[65,75]$. So, differently from the case of non-flexible timelines, durations alone are not sufficient to suitably represent tokens. Analogously, end time bounds do not capture all the necessary information: the above described token $x^{i}$ does not represent a valued interval starting at $40$ and ending at $75$, even though it respects the start and end time bounds, it violates the duration constraint.

Controllability tags are part of token structures for a different reason. Although $\gamma(x^{i})$ is equal to $\gamma(\operatorname{val}(x^{i}))$, such information is included in the token $x^{i}$ with the aim of having a self-contained representation of flexible plans, encapsulating all the relevant execution information. This allows the executive system to handle plans with no need of considering also the description of the state variables. When considering a set FTL of timelines for the state variables in $SV$, it is always assumed that it contains exactly one timeline for each element of $SV$.

Schedules

A scheduled timeline is a particular case where each token has a singleton $[t,t]$ as its end time, i.e. the end times are all fixed. A schedule of a timeline $\mathit{FTL}_{x}$ is essentially obtained from $\mathit{FTL}_{x}$ by narrowing down to singletons (i.e. time points) the token end times.

The schedule of a token corresponds to one of the valued intervals it represents (i.e., it is obtained by choosing an exact end point in the allowed interval, without changing its duration bounds). A scheduled timeline is a sequence of scheduled tokens satisfying the duration requirements. Tokens, timelines and sets of timelines represent the set of their schedules.

In general, $\mathit{STL}_{x}$ and STL will be used as meta-variables for scheduled timelines and sets of scheduled timelines, respectively, while $\mathit{FTL}_{x}$ and FTL as meta-variables for generic (flexible) timelines and sets of timelines. In what follows, an interval of the form $[t,t]$, consisting of a single time point, will be identified with the time point $t$ (and, with an abuse of notation, singleton intervals are allowed as operands of additions, subtractions, comparison operators, etc.).

Definition 4.2.6**.**

[Scheduled]* A scheduled token is a token of the form*

[TABLE]

*(or succinctly $x^{i}=(v,t,[d,d^{\prime}],\gamma(v))$). A schedule of a token $x^{i}=(v,\allowbreak[e,e^{\prime}],\allowbreak[d,d^{\prime}],\gamma(v))$ is a scheduled token $x^{i}=(v,t,[d,d^{\prime}],\gamma(v))$, where $e\leq t\leq e^{\prime}$.

A scheduled timeline $\mathit{STL}_{x}$ is a timeline consisting only of scheduled tokens and such that if $k$ is the timeline length, then: for all $1\leq i\leq k$, if $\operatorname{duration}(x^{i})=[d_{i},d^{\prime}_{i}]$, then $d_{i}\leq\operatorname{end\_time}(x^{i})-\operatorname{start\_time}(x^{i})\leq d^{\prime}_{i}$.

A scheduled timeline $\mathit{STL}_{x}$ for the state variable $x$ is a schedule of $\mathit{FTL}_{x}$ if $\mathit{STL}_{x}$ and $\mathit{FTL}_{x}$ have the same length $k$, and for all $i$, $1\leq i\leq k$, the token $x^{i}$ of $\mathit{STL}_{x}$ is a schedule of the token $x^{i}$ of $\mathit{FTL}_{x}$.

Let FTL be a set of timelines for the state variables in $SV$. A schedule STL of FTL is a set of scheduled timelines for the state variables in $SV$, where each $\mathit{STL}_{x}\in\mbox{\bf STL}$ is a schedule of the timeline $\mathit{FTL}_{x}\in\mbox{\bf FTL}$.*

In simple terms, a scheduled timeline is a timeline where every end time is a singleton respecting the duration bounds. A schedule of a timeline is a way of assigning values to each token end time, so that both duration and end time bounds are respected. Tokens, timelines, and sets of timelines represent the set of their respective schedules.

Example 4.2.7**.**

Let us consider the flexible timeline $\mathit{FTL}_{inst}$ of example 4.2.5:

[TABLE]

*It is worth pointing out that, since the start time of the first token of a timeline is $[0,0]$, its end time bounds are usually equal to its duration bounds, but, like this example shows, it is not necessarily so.

When the two intervals differ, the end point of the corresponding first token in any schedule of the timeline belongs to their intersection. Each schedule of the timeline $\mathit{FTL}_{inst}$ represents a series of choices for the token end points, within the allowed intervals and respecting the allowed durations.

For instance, the following timeline is a schedule of $\mathit{FTL}_{inst}$:*

[TABLE]

*In fact, it satisfies all of the endpoint and duration bounds in $\mathit{FTL}_{inst}$.

Clearly, not every sequence of scheduled tokens is a scheduled timeline. For instance, the sequence of tokens obtained from $\mathit{STL}_{inst}$ by replacing the token $inst^{2}$ with $inst^{2}=(Unstowing,31,[3,3],c)$ is not a scheduled timeline at all, since it does not satisfy the duration constraints for $inst^{2}$: $\operatorname{end\_time}(inst^{2})-\operatorname{start\_time}(inst^{2})=31-25=6>3$.

Let us now consider the scheduled timeline $\mathit{STL}^{\prime}_{inst}$ obtained from $\mathit{STL}_{inst}$ by replacing the token $inst^{3}$ with $inst^{3}=(Unstowed,60,[19,32],c)$. Although $\mathit{STL}^{\prime}_{inst}$ satisfies all the duration bounds, it is not a schedule of $\mathit{FTL}_{inst}$, since the end time of $inst^{3}$ is not in the allowed interval $[50,55]$ of $inst^{3}$ in $\mathit{FTL}_{inst}$.*

4.2.2 Restricting the Behavior of State Variables

The behavior of state variables may be restricted by requiring that time intervals with given state variable values satisfy some temporal constraints. For instance, in the Rover sample domain, data can be communicated only when the communication channel is available. In other terms, for every token $cm^{i}$ in the timeline for the state variable $cm$ having the value $Communicating$, there must exist a token in the timeline for $win$, with the value $Available$ and bearing a given temporal relation with $inst^{i}$. This type of relations are expressed by means of synchronization rules that complete the definition of all the components of a domain specification.

Temporal Relations

As a first step, the set of allowed temporal relations is introduced. They are either relations between two intervals or relations between an interval and a time point. In particular, this work considers quantitative temporal constraints. For the sake of simplicity, a small set of primitive relations is chosen, all of which are parametrized by a (single) temporal interval.

Definition 4.2.8**.**

[Temporal Relation]* A temporal relation between intervals is an expression of the form $A$ $\rho_{[lb,ub]}$ $B$, where $A$ $=$ $[s_{A},e_{A}]$ and $B$ $=$ $[s_{B},e_{B}]$ are time intervals, with $s_{A},\allowbreak e_{A},\allowbreak s_{B},\allowbreak e_{B}\in\mathbb{T}$, $\rho_{[lb,ub]}\in\mbox{\bf R}$ $=$ $\{\,\operatorname{start\_before\_start},\allowbreak\,\operatorname{end\_before\_end},\allowbreak\,\operatorname{start\_before\_end},\allowbreak\,\operatorname{end\_before\_start}\}$, $lb\in\mathbb{T}$ and $ub\in\mathbb{T}^{\infty}$. The following table defines when a relation $A$ $\rho_{[lb,ub]}$ $B$ holds:*

[TABLE]

Other relations can be defined in terms of the primitive relations in Definition 4.2.8 (and their converses). These relations, like those used by EUROPA [Barreiro et al., 2012] and APSI-TRF [Fratini et al., 2011], correspond to the quantitative extension of Allen’s temporal relations [Allen, 1983]. Thus, the relations in the left most column of Table 4.1, are meant as abbreviations of the corresponding expressions on their right.

Once relations on time intervals are defined, they can be transposed to relations on tokens. The expressions used to denote such relations refer to tokens by means of their names.

Definition 4.2.9**.**

[Token Relation]* Let $x^{i}$ and $y^{j}$ be names of tokens belonging to scheduled timelines for the state variables $x$ and $y$, respectively, with $\operatorname{start\_time}\left(x^{i}\right)$ $=$ $s_{i}$, $\operatorname{end\_time}\left(x^{i}\right)$ $=$ $e_{i}$, $\operatorname{start\_time}\left(y^{j}\right)$ $=$ $s_{j}$, $\operatorname{end\_time}\left(y^{j}\right)$ $=$ $e_{j}$. Moreover, let $t,lb\in\mathbb{T}$ and $ub\in\mathbb{T}^{\infty}$. Expressions of the form $x^{i}\,\rho_{[lb,ub]}y^{j}$, for $\rho\in\mbox{\bf R}$, and $x^{i}\,\rho_{[lb,ub]}t$, for $\rho\in\mbox{\bf R}^{\prime}$, are called relations on tokens.

The relation $x^{i}$ $\rho_{[lb,ub]}$ $y^{j}$ holds iff $[s_{i},e_{i}]$ $\rho_{[lb,ub]}$ $[s_{j},e_{j}]$ holds and the relation $x^{i}$ $\rho_{[lb,ub]}$ $t$ holds iff $[s_{i},e_{i}]$ $\rho_{[lb,ub]}$ $t$ holds. When a relation on two tokens $x^{i}$ and $y^{j}$ holds, we also say that the tokens $x^{i}$ and $y^{j}$ satisfy the relation, and that any set of scheduled timelines that contain $x^{i}$ and $y^{j}$ satisfies the relation. Analogously, if a relation $x^{i}\,\rho_{[lb,ub]}t$ holds, then the token $x^{i}$ and any set of scheduled timelines containing $x^{i}$ satisfy the relation.*

Example 4.2.10**.**

Let $\mbox{\bf STL}=\{\mathit{STL}_{cm},\mathit{STL}_{win}\}$ be a set of timelines for the Rover domain, including $\mathit{STL}_{r}$, $\mathit{STL}_{inst}$ and $\mathit{STL}_{nav}$, where $\mathit{STL}_{cm}$ contains the tokens

[TABLE]

and $\mathit{STL}_{win}$ contains the tokens

[TABLE]

The expressions $win^{2}\,\operatorname{start\_before\_start}_{[5,\infty]}cm^{6}$ and $cm^{6}\operatorname{ends\_before}_{[30,45]}165$ are relations on tokens and they are satisfied by STL.

Synchronization Rules

A synchronization constraint can be informally considered as a statement of the form "for every token …there exist tokens such that …". Namely, it represents a kind of quantified sentence. The formal counterpart of this kind of assertions makes use of variables: for every $var_{0}$ …there exist $var_{1},\dots var_{n}$ such that …. The variables used to express synchronizations are called token variables. They are taken from a (potentially infinite) set $X=\{a_{0},a_{1},\dots\}$ of names, whose elements are all different from variable names, values and numbers. These variables are intended to range over tokens in the considered set of timelines.

Making a step forward, it can be observed that synchronization assertions actually use a form of bounded quantification: "for all/exist tokens with value $v$ in the timeline for the state variable $x$ …". Such token variables with restricted range will be denoted by expressions of the form $a_{i}[x=v]$, where $a_{i}$ is a token variable, $x$ is a state variable name, and $v\in\operatorname{values}(x)$. Such expression are called annotated token variables. The next definition introduces the form of the assertions that can be used to express parametrized relations on tokens.

Definition 4.2.11**.**

[Existential Statement]* An atom is either the special constant $\top$ or an expression of the form $a_{i}$ $\rho_{[lb,ub]}$ $a_{j}$ or $a_{i}$ $\rho^{\prime}_{[lb,ub]}$ $t$, where $a_{i}$ and $a_{j}$ are token variables, $lb,t\in\mathbb{T}$, $ub\in\mathbb{T}^{\infty}$, $\rho\in\mbox{\bf R}$, and $\rho^{\prime}\in\mbox{\bf R}^{\prime}$.

An existential statement is an expression of the form*

[TABLE]

where

• (i)

$a_{1},\dots,a_{n}$* are distinct token variables;*

• (ii)

for all $i$ $=$ $1,\dots,n$, $x_{i}$ is a state variable and $v_{i}\in\operatorname{values}(x_{i})$ (i.e., $a_{i}[x_{i}=v_{i}]$ is an annotated token variable);

• (iii)

$\cal C$* is a conjunction of atoms.*

The bound variables of the statement are $a_{1},\dots,a_{n}$ and any variable different from $a_{1},\dots,a_{n}$ possibly occurring in $\cal C$ is said to occur free in the statement.

Disjunctions of existential statements constitute the body of synchronization rules.

Definition 4.2.12**.**

[Synchronization Rule]* A synchronization rule is an expression of the form*

[TABLE]

*(for $k\geq 1$) where every ${\cal E}_{i}$ is an existential statement whose bound variables are all different from $a_{0}$ and where only the token variable $a_{0}$ may occur free. The left-hand part of the synchronization rule, $a_{0}[x_{0}=v_{0}]$, is called the trigger of the rule.

A synchronization rule with empty trigger is an expression of the form:*

[TABLE]

(for $k\geq 1$) where every ${\cal E}_{i}$ is an existential statement with no free variables.

Intuitively, a synchronization rule with non-empty trigger of the above form requires that, whenever the state variable $x_{0}$ assumes the value $v_{0}$ in some interval $a_{0}$, there is at least an existential statement {\cal E}_{i}=\exists\,a_{1}[x_{1}=v_{1}]\dots a_{n}[x_{n}=v_{n}]\,.\,\mbox{\cal C} and tokens $a_{i}$ ($1\leq i\leq n$) where the variable $x_{i}$ has the value $v_{i}$, such that $\cal C$ holds (if \mbox{\cal C}=\top, no temporal relation is required to hold). When the trigger is empty, the existence of the intervals $a_{i}$ and the relations among them have to hold unconditionally. Synchronization rules with empty triggers are useful to represent domain invariants, as well as planning goals (both called "facts" in [Cimatti et al., 2013]). The use of token variables (which are absent in [Cimatti et al., 2013]) allows one to refer to different intervals having the same value. Indeed, although the token variables $a_{0},\dots,a_{n}$ are pairwise distinct, multiple occurrences of state variable names and values are allowed.

Example 4.2.13**.**

Consider the operational constraint of the Rover domain concerning the data communication activity: the rover can send data only when the communication channel is available and when it is not moving. A synchronization rule expressing this operational constraint is the following:

[TABLE]

According to this rule, whenever the state variable $cm$ assumes the value SendData in an interval $a_{0}$, the state variable $win$ has the value Available in some interval $a_{1}$ containing $a_{0}$, and the state variable $nav$ must have the value At in some interval $a_{2}$ containing $a_{0}$ or the value Home in some interval $a_{2}$ containing $a_{0}$. Namely the rover cannot move during communication tasks and communications can be performed only if the communication channel is available. Synchronization rules with empty triggers may be useful to state known facts, such as, for instance:

[TABLE]

This rule represents the fact that the rover is at the "home" location at the beginning of the mission. Synchronization rules with empty triggers are also used to represent planning goals, as will be described later on.

The following definition introduces the semantics of synchronization on scheduled timelines. Since the statement of a synchronization rule makes use of token variables, each of them must be "interpreted", i.e., mapped to a token of the considered timelines.

Definition 4.2.14**.**

[Satisfiability of Synchronization Rules]* Let FTL be a set of timelines for the state variables $SV$. A token assignment for a set of annotated token variables $\{a_{1}[x_{1}=v_{1}],\dots,a_{n}[x_{n}=v_{n}]\}$ on FTL is a function $\varphi$ mapping every $a_{i}$ to a token of the timeline $\mathit{FTL}_{x_{i}}\in\mbox{\bf FTL}$ and such that $\operatorname{val}\left(\varphi\left(a_{i}\right)\right)$ $=$ $v_{i}$ for all $i=1,\dots,n$.

Let \mbox{\cal C}=A_{1}\wedge\dots\wedge\ A_{m} be a conjunction of atoms and STL a set of scheduled timelines, including a timeline for every state variable occurring in $\cal C$. A token assignment $\varphi$ on STL satisfies $\cal C$ if for every atom $A\in\{A_{1},\dots,A_{m}\}$,*

• (i)

if $A$ $=$ $a_{i}$ $\rho_{[lb,ub]}$ $a_{j}$ then the relation $\varphi\left(a_{i}\right)$ $\rho_{[lb,ub]}$ $\varphi\left(a_{j}\right)$ holds;

• (ii)

if $A$ $=$ $a_{i}$ $\rho_{[lb,ub]}$ $t$, then the relation $\varphi\left(a_{i}\right)$ $\rho_{[lb,ub]}$ $t$ holds.

A token assignment $\varphi$ on STL satisfies an existential statement of the form

[TABLE]

if $\varphi$ is a token assignment for a set of annotated variables including $a_{0}[x_{0}=v_{0}],\dots,a_{n}[x_{n}=v_{n}]$ and $\varphi$ satisfies $\cal C$. Let

[TABLE]

*be a synchronization rule. A set of scheduled timelines STL for the state variables $SV$ satisfies the synchronization rule $S$ if for every token $x_{0}^{k}$ in $\mathit{STL}_{x_{0}}\in\mbox{\bf STL}$ such that $\operatorname{val}(x_{0}^{k})=v_{0}$, there exists a token assignment $\varphi$ on STL such that $\varphi(a_{0})=x_{0}^{k}$ and $\varphi$ satisfies ${\cal E}_{i}$ for some $i\in\{1,\dots,k\}$.

A set of timelines STL for the state variables $SV$ satisfies a synchronization rule with empty trigger $\top\rightarrow{\cal E}_{1}\vee\dots\vee{\cal E}_{k}$ if, for some $i\in\{1,\dots,k\}$, there exists a token assignment $\varphi$ on STL satisfying ${\cal E}_{i}$. Let $SV$ be a set of state variables, $\cal S$ be a set of synchronization rules concerning variables in $SV$ and STL be a set of scheduled timelines for the state variables in $SV$. STL satisfies the set of synchronizations $\cal S$ iff STL satisfies all the elements of $\cal S$.*

Example 4.2.15**.**

Consider the synchronization rule given in Example 4.2.13, that constrains the rover to send data only when the communication channel is available, and to be still during communication:

[TABLE]

*Let $\mathit{STL}_{cm}$ and $\mathit{STL}_{win}$ be two scheduled timelines.

Assume that the timeline for the pointing system contains a single token whose value is $SendData$, and has the form:*

[TABLE]

In addition, assume that the timeline $\mathit{STL}_{win}$ (for the availability of the communication channel) and the timeline $\mathit{STL}_{nav}$ (for the navigation system of the rover) have the forms:

[TABLE]

The synchronization rule is satisfied by the scheduled timelines $\{\mathit{STL}_{cm}$, $\mathit{STL}_{win}$, $\mathit{STL}_{nav}\}$. In fact, $cm^{i}$ is the only token in $\mathit{STL}_{cm}$ whose value is $SendData$, and the token assignment $\varphi$, such that $\varphi(a_{0})=cm^{i}$ and $\varphi(a_{1})=win^{j}$, satisfies the existential statement $\exists\,a_{1}[win=Available]\,.\leavevmode\nobreak\ a_{1}\,\operatorname{contains}_{[0,\infty][0,\infty]}a_{0}$: $\operatorname{val}(\varphi(a_{1}))=Available$ and $\varphi$ satisfies $a_{1}\,\operatorname{contains}_{[0,\infty][0,\infty]}a_{0}$. The latter assertion holds because $\varphi(a_{1})\operatorname{contains}_{[0,\infty][0,\infty]}\varphi(a_{0})$ – i.e., $win^{j}\leavevmode\nobreak\ \operatorname{contains}_{[0,\infty][0,\infty]}\leavevmode\nobreak\ cm^{i}$ –  holds, since $[80,170]\leavevmode\nobreak\ \operatorname{contains}_{[0,\infty][0,\infty]}\leavevmode\nobreak\ [110,130]$ holds.

4.2.3 Planning Domains

A planning domain is described by specifying a set of state variables and a set of synchronization rules. The formal definition of planning domains is given next, together with the notion of a set of scheduled timelines respecting the requirements of the domain.

Definition 4.2.16**.**

[Planning Domain]* A planning domain is a triple (SV_{P},SV_{E},\mbox{\cal S}), where:*

• •

$SV_{P}$* is a set of planned state variables;*

• •

$SV_{E}$* is a set of external state variables (with $SV_{P}\cap SV_{E}=\emptyset$);*

• •

$\cal S$* is a set of synchronization rules involving state variables in $SV_{P}\cup SV_{E}$.*

A set of scheduled timelines STL for the state variables in $SV$ is valid with respect to the planning domain \mbox{\cal D}=(SV_{P},SV_{E},\mbox{\cal S}) if $SV=SV_{P}\cup SV_{E}$ and STL satisfies the set of synchronizations $\cal S$.

Example 4.2.17**.**

Let us consider the planning domain \mbox{\cal D}=(SV_{P},SV_{E},\mbox{\cal S}\} where $SV_{P}=\{r,inst,nav,cm\}$, $SV_{E}=\{win\}$ – for the state variables described in Example 4.2.3 – and $\cal S$ contains the following synchronization rules, modeling the operational constraints described in Section 4.1:

[TABLE]

Let moreover STL be the set of timelines containing

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

The set STL is valid with respect to the planning domain $\cal D$: it contains exactly one timeline for each state variable in $SV_{P}\cup SV_{E}$ and it satisfies all the synchronization rules of the domain.

4.3 Flexible Plans

The main component of a flexible plan is a set FTL of timelines, representing different sets $\mbox{\bf STL}_{i}$ of scheduled timelines. It may be the case that not every $\mbox{\bf STL}_{i}$ satisfies the synchronization rules of the domain. We aim at defining plans so that they contain all the information needed to execute them, without having to check how the behavior of state variables and timelines is constrained by the planning domain.111For the same reason controllability tags are included in token descriptions. Consequently, a flexible plan $\Pi$ must be equipped with additional information in order to guarantee that every set of scheduled timelines is valid with respect to the domain specification. Such information represent temporal relations that have to hold in order to satisfy the synchronization rules of the domain.

As a schematic example showing why a set of timelines does not convey enough information to represent a flexible plan, let us consider a domain with a synchronization rule $S$ of the form $a_{0}[x=v]\rightarrow\exists a_{1}[y=v^{\prime}].\,a_{0}\,\operatorname{meets}\,a_{1}$ and timelines for the state variables $x$ and $y$ containing, respectively, the tokens $x^{i}=(v,[30,50],[20,30],\gamma(v))$ and $y^{j}$, with $\operatorname{val}(y^{j})=v^{\prime}$ and $\operatorname{start\_time}(y^{j})=[30,50]$. Not every pair of schedules of $x^{i}$ and $y^{j}$ satisfies $S$. Thus, the representation of a flexible plan must also include information about the relations that must hold between tokens in order to satisfy the synchronization rules of the planning domain. In the example above, it would include the relation $x^{i}\,\operatorname{meets}\,y^{j}$. In general, a flexible plan includes a set of relations on tokens.

When there are different ways to satisfy a synchronization rule by the same set FTL of flexible timelines, there are also different (valid) flexible plans with the same set of timelines FTL. Thus, a flexible plan represents the set of its instances.

Definition 4.3.18**.**

[Flexible Plan]* A flexible plan $\Pi$ is a pair (\mbox{\bf FTL},\mbox{\cal R}), where FTL is a set of timelines and $\cal R$ is a set of relations on tokens involving token names in some timelines in FTL. An instance of the flexible plan \Pi=(\mbox{\bf FTL},\mbox{\cal R}) is any schedule of FTL that satisfies every relation in $\cal R$.*

In order to determine when a plan is valid with respect to a planning domain, the semantics of synchronizations on flexible plans must be defined. Essentially, a plan \Pi=(\mbox{\bf FTL},\mbox{\cal R}) satisfies a synchronization rule $S$ if the constraints represented by $S$ are guaranteed to hold for any schedule of FTL satisfying the relations in $\cal R$. In other terms, $\cal R$ represents a possible way to satisfy $S$. The intuition underlying the formal definition can be explained as follows.

When considering a plan \Pi=(\mbox{\bf FTL},\mbox{\cal R}), a mapping is used to assign the annotated token variables occurring in the synchronization to token names occurring in FTL. Let us consider, for instance a rule of the form

[TABLE]

Let us moreover assume that $x^{3}$ is a token in the timeline for $x$ in FTL with $\operatorname{val}(x^{3})=v$, and that the timeline for $y$ in FTL contains exactly two tokens $y^{5}$ and $y^{8}$ having value $v^{\prime}$. In order for the rule to be satisfied, FTL must be constrained by requiring that $x^{3}$ starts from 10 to 20 time units after the end of either $y^{5}$ or $y^{8}$. The plan $\Pi$ commits to one of the two alternatives: binding $a_{1}$ to either $y^{5}$ or $y^{8}$.

Example 4.3.19**.**

Let \Pi=(\mbox{\bf FTL},\mbox{\cal R}), where:

• •

FTL* contains the timelines*

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

• •

$\cal R$* contains the temporal relations*

[TABLE]

$\Pi$* is a flexible plan, and the set STL of scheduled timelines containing*

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

*is an instance of $\Pi$, since $\{\mathit{STL}_{r}$, $\mathit{STL}_{inst}$, $\mathit{STL}_{nav}$, $\mathit{STL}_{cm}$, $\mathit{STL}_{win}\}$ is a schedule of FTL and it satisfies all the temporal relations in $\cal R$.

If however, the end time of $inst^{5}$ in $\mathit{STL}_{inst}$ is replaced by $68$, the so obtained set of scheduled timelines is not an instance of $\Pi$, although it is a schedule of FTL, because the relation*

[TABLE]

is not satisfied.

The correspondence between token variables and token names is established by use of a function $\varphi$ mapping $a_{1}$ to either $y^{5}$ or $y^{8}$. According to the chosen option, the set of relations $\cal R$ in the plan contains either $y^{5}\,\operatorname{end\_before\_start}_{[10,20]}x^{3}$ or $y^{8}\,\operatorname{end\_before\_start}_{[10,20]}x^{3}$.

Definition 4.3.20**.**

[Plan Satisfiability]* Let \mbox{\cal C}=A_{1}\wedge\dots\wedge A_{m} be a conjunction of atoms, \Pi=(\mbox{\bf FTL},\mbox{\cal R}) a flexible plan, where FTL contains a timeline for every state variable occurring in $\cal C$, and $\varphi$ a token assignment on FTL. The plan $\Pi$ satisfies $\cal C$ with $\varphi$ if for every atom $A\in\{A_{1},\dots,A_{m}\}$, (i) if $A$ $=$ $a_{i}$ $\rho_{[lb,ub]}$ $a_{j}$, then $\varphi\left(a_{i}\right)$ $\rho_{[lb,ub]}$ \varphi\left(a_{j}\right)\in\mbox{\cal R}, and (ii) if $A$ $=$ $a_{i}$ $\rho_{[lb,ub]}$ $t$, then $\varphi\left(a_{i}\right)$ $\rho_{[lb,ub]}$ t\in\mbox{\cal R}. Let*

[TABLE]

*be an existential statement and $\varphi$ be a token assignment on FTL. The flexible plan $\Pi$ satisfies $\cal E$ with $\varphi$ if $\varphi$ is an assignment for a set of annotated token variables including $a_{0}[x_{0}=v_{0}],\dots,a_{n}[x_{n}=v_{n}]$ and $\Pi$ satisfies $\cal C$ with $\varphi$.

The plan $\Pi$ satisfies a synchronization rule with non-empty trigger*

[TABLE]

*if for every flexible token $x_{0}^{m}$ of the timeline $\mathit{FTL}_{x_{0}}\in\mbox{\bf FTL}$ such that $\operatorname{val}(x_{0}^{m})=v_{0}$, there exists a token assignment $\varphi$ on FTL such that $\varphi(a_{0})=x_{0}^{m}$ and $\Pi$ satisfies ${\cal E}_{i}$ with $\varphi$, for some $i\in\{1,\dots,k\}$.

The plan $\Pi$ satisfies a synchronization rule with empty trigger*

[TABLE]

if, for some $i\in\{1,\dots,k\}$, there exists a token assignment $\varphi$ on FTL such that $\Pi$ satisfies ${\cal E}_{i}$ with $\varphi$.

Example 4.3.21**.**

Let us consider the flexible timelines $\mathit{FTL}_{cm}$, $\mathit{FTL}_{nav}$ and $\mathit{FTL}_{win}$ of Example 4.3.19. The flexible plan \Pi=(\mbox{\bf FTL},\mbox{\cal R}), where $\mbox{\bf FTL}=\{$ $\mathit{FTL}_{cm}$, $\mathit{FTL}_{nav}$, $\mathit{FTL}_{inst}$, $\mathit{FTL}_{win}\}$ and $\cal R$ containing the temporal relations

[TABLE]

satisfies the synchronization rule with trigger SendData, given in Example 4.2.17:

[TABLE]

Considering the definition of the relation $\operatorname{contains}$ given in Table 4.1, the two temporal constraints of the synchronization rule can be rewritten as:

[TABLE]

[TABLE]

The set $\cal R$ contains the atoms

[TABLE]

[TABLE]

*for the token assignment $\varphi$ such that $\varphi(a_{0})=cm^{2}$, $\varphi(a_{1})=win^{2}$ and $\varphi(a_{2})=nav^{3}$. Clearly, there might be schedules of the set of timelines FTL that do not satisfy all the requirements. For example the requirement $win^{2}\operatorname{contains}_{[0,\infty]}{[0,\infty]}cm^{2}$ is not satisfied by the schedules where $\operatorname{start\_time}(cm^{2})=70$, $\operatorname{end\_time}(cm^{2})=108$, $\operatorname{start\_time}(win^{2})=30$ and $\operatorname{end\_time}(win^{2})=90$, which consequently are not instances of the flexible plan $\Pi$.

However, if schedules like those in Example 4.3.19 satisfy all the requirements, then such schedules of FTL are also instances of $\Pi$. As a further example showing how a plan commits to a choice among the possibly different ways to satisfy a synchronization rule, let us consider a set FTL of timelines and a rule $S$ of the form*

[TABLE]

*Let us moreover assume that $x^{3}$ and $x^{7}$ are the only tokens with value $v$ in the timeline $\mathit{FTL}_{x}$ for $x$ in FTL, that the timeline $\mathit{FTL}_{y}$ for $y$ in FTL contains exactly one token $y^{5}$ having value $v^{\prime}$, and that $\mathit{FTL}_{z}$ contains exactly one token $z^{8}$ with value $v^{\prime\prime}$.

In order to satisfy the rule $S$:*

$\cal R$* must contain the constraints*

[TABLE]

In fact the plan has to satisfy either $\exists a_{1}[y=v^{\prime}].a_{1}\,\operatorname{end\_before\_start}_{[10,20]}a_{0}$ or $\exists a_{1}[z=v^{\prime\prime}].a_{0}\,\operatorname{end\_before\_start}_{[5,\infty]}a_{1}$ with a token assignment $\varphi$ such that $\varphi(a_{0})=x^{3}$. If $\cal R$ contains $y^{5}$ $\,\operatorname{end\_before\_start}_{[10,20]}$ $x^{3}$, then the plan satisfies the existential statement (i) with $\varphi$, when $\varphi(a_{1})=y^{5}$. If it contains $x^{3}$ $\,\operatorname{end\_before\_start}_{[5,\infty]}$ $z^{8}$, then the plan satisfies (ii) with $\varphi$, when $\varphi(a_{1})=z^{8}$. 2. 2.

$\cal R$* must contain the constraints*

[TABLE]

The reasoning is the same as above, just replacing $x^{7}$ for $x^{3}$.

Therefore, for instance, both plans

[TABLE]

satisfy $S$

The notions of plan validity and consistency can now be defined.

Definition 4.3.22**.**

[Plan Validity]* A flexible plan \Pi=\left(\mbox{\bf FTL},\mbox{\cal R}\right) is valid with respect to a planning domain \mbox{\cal D}=(SV_{P},SV_{E},\mbox{\cal S}) iff:*

FTL* is a set of timelines for the state variables $SV=SV_{P}\cup SV_{E}$;* 2. 2.

$\Pi$* satisfies all the synchronization rules in $\cal S$;* 3. 3.

for each planned state variable $x=(V,T,\gamma,D)\in SV_{P}$, and each uncontrollable token $x^{i}$ in $\mathit{FTL}_{x}\in\mbox{\bf FTL}$, if $D(\operatorname{val}(x^{i}))=(d_{min},d_{max})$ and $\operatorname{start\_time}(x^{i})=[s,s^{\prime}]$, then $\operatorname{duration}(x^{i})$ = $[d_{min},d_{max}]$ and $\operatorname{end\_time}(x^{i})$ = $[s+d_{min},s^{\prime}+d_{max}]$.

The plan $\Pi$ is consistent if there exists at least one instance of $\Pi$.

The last condition required for a plan to be valid guarantees that the plan does not make any hypothesis on the duration of uncontrollable values of planned variables. The restriction is not applied to external variables, since the planner is not allowed to control them at all: their behavior is described in the planning problem as a sort of observation of the external world.

It is important to point out that plan consistency is a minimal requirement for a plan to be considered meaningful, although, when the domain includes uncontrollable elements, it is not enough to guarantee its executability. In this regards, the work [Cialdea Mayer et al., 2016] proves a result showing that there exists a set $\Theta$ of flexible plans for which an effective consistency check procedure exists, yet every scheduled valid plan is an instance of some flexible plan in $\Theta$. Intuitively, each plan $\Pi\in\Theta$ is such that the sequence of scheduled tokens, obtained by fixing every token end point to the lower bound of the respective end time interval, is an instance of $\Pi$ (i.e., it is a scheduled timeline respecting the relations in $\Pi$). The mentioned result implies that, when searching for a consistent plan, it is sufficient to consider candidate plans in $\Theta$, respecting the above condition.

4.4 Problem Specification

In timeline-based planning, a planning problem typically includes a planning horizon, i.e., the time by which the system behavior has to be planned. Finally, since the external state variables are not under the system control, the problem must include information about their behavior up to the given horizon. Such information is given in the form of a set of flexible timelines and temporal relations on their tokens.

Definition 4.4.23**.**

[Planning Problem]* A planning problem is a tuple (\mbox{\cal D},\mbox{\cal G},\mbox{\cal O},H), where \mbox{\cal D}=(SV_{P},\allowbreak SV_{E},\allowbreak\mbox{\cal S}) is a planning domain, $\cal G$ a planning goal for $\cal D$, $H\in\mathbb{T}_{>0}$ is the planning horizon, and \mbox{\cal O}=(\mbox{\bf FTL}_{E},\mbox{\cal R}_{E}), where*

• (i)

$\mbox{\bf FTL}_{E}$* is a set containing exactly one flexible timeline for each external state variable in $SV_{E}$;*

• (ii)

the horizon of every timeline in $\mbox{\bf FTL}_{E}$ is $[h,h^{\prime}]$ for some $h\geq H$;

• (iii)

\mbox{\cal R}_{E}* is a set of temporal relations on tokens of timelines in $\mbox{\bf FTL}_{E}$;*

• (iv)

(\mbox{\bf FTL}_{E},\mbox{\cal R}_{E})* is consistent, i.e., there is at least one schedule of $\mbox{\bf FTL}_{E}$ satisfying the relations in \mbox{\cal R}_{E}.*

The pair $\cal O$, called the observation, specifies the behavior of external state variables up to a time point not less than the planning horizon. Item (iv) rules out inconsistent observations, i.e., descriptions of the behavior of the external state variables with no instances. The pair (\mbox{\bf FTL}_{E},\mbox{\cal R}_{E}) can be viewed as a flexible plan. In particular, even when \mbox{\cal R}_{E}=\O, the set of timelines $\mbox{\bf FTL}_{E}$ must have at least one schedule.

The planner must respect what is specified by the set of timelines $\mbox{\bf FTL}_{E}$, without taking any autonomous decision: this requirement is fulfilled simply when the timeline for each external variable in the plan is exactly the timeline for the same state variable in $\mbox{\bf FTL}_{E}$. The relations in \mbox{\cal R}_{E} represent known facts about the external world.

It is worth pointing out that, since $\mbox{\bf FTL}_{E}$ is a set of timelines, the planner knows how the external components evolve, i.e., the sequence of activities/states constituting their behavior, the only uncertainty being the duration of such states. This rules out, for instance, scenarios where the uncontrollable events might occur an unknown number of times within the given horizon.

Example 4.4.24**.**

For instance, a planning problem for our sample domain can be the problem \Pi=(\mbox{\cal D},\mbox{\cal G},\mbox{\cal O},H), where

• •

$\cal D$* is the planning domain of Example 4.2.3, i.e., \mbox{\cal D}=(SV_{P},SV_{E},\mbox{\cal S}\} where $SV_{P}=\{r,inst,nav,cm\}$, $SV_{E}=\{win\}$ and $\cal S$ contains the synchronization rules of Example 4.2.17*

• •

\mbox{\cal G}=(\Gamma,\Delta)* – see Example 4.4.26 – where*

[TABLE]

and

[TABLE]

• •

\mbox{\cal O}=(\{\mathit{FTL}_{win}\},\mbox{\cal R}_{E}), where

[TABLE]

• •

$H=200$.

4.4.1 Planning Goals

A planning problem includes the description of the underlying planning domain and of a desired goal to be accomplished. This work considers temporally extended goals: a planning goal specifies that some planned variables have to assume some given values in some intervals, possibly satisfying some temporal relations. Disjunctive goals are also allowed.

Definition 4.4.25**.**

[Planning Goal]* A planning goal $\cal G$ for a domain \mbox{\cal D}=(SV_{P},SV_{E},\mbox{\cal S}) is a pair $(\Gamma,\Delta)$, where:*

• (i)

$\Gamma$* is a set of accomplishment goals, i.e., expressions of the form $g=(x,v)$, where $g$ is a token variable, called the goal name, $x\in SV_{P}$, and $v\in\operatorname{values}(x)$;*

• (ii)

$\Delta$, called a relational goal, is a disjunction \mbox{\cal D}_{1}\vee\dots\vee\mbox{\cal D}_{k}, where each \mbox{\cal D}_{i} is a conjuntion of atoms containing only goal names occurring in $\Gamma$.

A planning goal \mbox{\cal G}=(\Gamma,\Delta), with $\Gamma=\{g_{1}=(x_{1},v_{1}),\allowbreak\dots,\allowbreak g_{n}=(x_{n},\allowbreak v_{n})\}$ and \Delta=\mbox{\cal D}_{1}\vee\dots\vee\mbox{\cal D}_{k}, is represented by a synchronization rule $S_{\cal G}$ with empty trigger, of the form:

[TABLE]

It is worth pointing out that restrictions on the start and end intervals of a given goal (like in [Cimatti et al., 2013, Cesta et al., 2009]) can be expressed by means of relational goals. In particular, if the start point of a given goal $g$ is required to be in the interval $[s,s^{\prime}]$ and its end point in $[e,e^{\prime}]$, then such restrictions can be expressed by the relational goal $(g\,\operatorname{starts\_after}_{[0,s^{\prime}-s]}s)\wedge(g\,\operatorname{ends\_after}_{[0,e^{\prime}-e]}e)$.

Example 4.4.26**.**

A simple planning goal for the Rover domain may be that, in order to accomplish the mission, the rover has to take a sample of a target to be analyzed and then communicate the scientific results no later than 65 time units after the completion of the sampling task. Such a goal is is represented by the pair $(\Gamma,\Delta)$, where

[TABLE]

and

[TABLE]

which can be turned into the synchronization rule

[TABLE]

Analogously, if the rover has to come back "home" (a known initial position) to complete the mission and we want to specify an alternative ordering constraint for the communication task, i.e., the rover may communicate scientific data either before going back "home" or immediately after, the goal is the goal is \mbox{\cal G}=(\Gamma,\Delta), where

[TABLE]

and

[TABLE]

The corresponding synchronization rule is:

[TABLE]

The next definition introduces the notion of goal fulfilment for scheduled timelines.

Definition 4.4.27**.**

[Goal Satisfiability]* A set of scheduled timelines STL fulfils the planning goal $\cal G$ if it satisfies the synchronization rule $S_{\cal G}$ representing $\cal G$.*

4.4.2 Solution Plans

Definition 4.4.28**.**

[Solution Plan]* Let \mbox{\cal P}=(\mbox{\cal D},\mbox{\cal G},\mbox{\cal O},H) be a planning problem and \Pi=(\mbox{\bf FTL},\mbox{\cal R}) be a flexible plan. $\Pi$ is a flexible solution plan for $\cal P$ if:*

for every planned state variable $x$, the horizon of $\mathit{FTL}_{x}\in\mbox{\bf FTL}$ is $[H,H]$; 2. 2.

$\Pi$* is valid with respect to $\cal D$;* 3. 3.

$\Pi$* satisfies the synchronization rule $S_{{\cal G}}$ representing $\cal G$;* 4. 4.

If \mbox{\cal O}=(\mbox{\bf FTL}_{E},\mbox{\cal R}_{E}), then $\mbox{\bf FTL}_{E}\subseteq\mbox{\bf FTL}$.

The first condition above guarantees that the behavior of the planned state variables is determined exactly up to the horizon of the planning problem, henceforth (condition 3) all the planning goals are achieved in due time. It is worth pointing out that condition 1 implies that the last token of each planned timeline must be controllable. Condition 4 ensures that the plan does not make any assumption on external variables, except for what is implied by the state variable definition and the observation.

Example 4.4.29**.**

Let us consider, for instance, the problem $\cal P$ of Example 4.4.24 and the flexible plan \Pi=(\mbox{\bf FTL},\mbox{\cal R}), where:

• •

$\mbox{\bf FTL}=\{\mathit{FTL}_{r},\mathit{FTL}_{inst},\mathit{FTL}_{nav},\mathit{FTL}_{cm},\mathit{FTL}_{win}\}$, where the timelines are those of the Example 4.3.19:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

• •

$\cal R$* contains the two relations on tokens*

[TABLE]

The plan $\Pi$ is a flexible solution plan for the planning problem $\cal P$ because:

• •

the horizon is 200 for all the timelines of the planned variables;

• •

$\Pi$* is valid with respect to $\cal D$:*

• –

FTL* contains the timelines for $r$, $inst$, $nav$, $cm$ and $win$;*

• –

$\Pi$* satisfies the synchronization rule of the domain; *

• –

all uncontrollable tokens satisfy the duration constraints of the related values

• •

$\Pi$* satisfies the synchronization rule $S_{{\cal G}}$ representing $\cal G$: the tokens $pm^{3}$ and $pm^{6}$ have values $Science$ and $Comm$, respectively, and $\cal R$ contains the relation $pm^{3}\operatorname{before}_{[0,65]}pm^{6}$.*

• •

$\mathit{FTL}_{gv}\in\mbox{\bf FTL}$.

The next result proves that information encoded by a flexible solution plan $\Pi$ for a given planning problem is sufficient to ensure that every instance of $\Pi$ is valid with respect to the planning domain and it fulfils the goal. Although the proof of this result is a straightforward consequence of the definitions, it deserves to be stated explicitly, since flexible plans without such a property would be meaningless.

Theorem 4.4.30**.**

If the plan $\Pi$ is a flexible solution plan for the problem

[TABLE]

then every instance of $\Pi$ is valid with respect to $\cal D$ and fulfils the goal $\cal G$.

Chapter 5 The Extensible Planning and Scheduling Library

Timeline-based applications are problem solvers capable of taking into account several features of a problem as well as integrating different techniques into the reasoning process (e.g. planning and scheduling integration). The design and implementation choices made to realize this kind of applications are really complex and closely connected to the specific characteristics of the particular problem to address. These choices are difficult to replicate and therefore it is not easy to leverage past experience and deploy existing applications to different contexts. Typically, it is necessary to start developing new applications from scratch in order to solve new types of problem.

The Extensible Planning and Scheduling Library (EPSL) [Umbrico et al., 2015, Cesta et al., 2013] is the result of a research effort which aims at realizing a general purpose timeline-based framework capable of supporting the design of P&S applications. The modeling features of EPSL concerning the representation and management of timelines take inspiration from APSI-TRF. Nevertheless, according to the formalization described in [Cialdea Mayer et al., 2016], EPSL extends the APSI-TRF representation by introducing temporal uncertainty in shape of uncontrollable activities and external features of a planning domain. Temporal uncertainty allows EPSL to address planning problems in which not all the features of the domain are under the control of the system. Thus, EPSL-based solvers generate, if possible, plans with some desired property concerning temporal controllability [Morris et al., 2001, Vidal and Fargier, 1999]. The validity of a plan with respect to the domain specification does not represent a sufficient condition to guarantee its executability in the real-world. Namely, the uncontrollable dynamics of the environment may prevent the complete and correct execution of plans. Thus, from the planning perspective, it is important to generate plans with some minimum controllability properties. EPSL takes into account the pseudo-controllability property which represents a necessary but not sufficient condition for dynamic controllability [Morris et al., 2001] of plans.

Broadly speaking, the solving approach is a general plan refinement procedure which iteratively detects a set of flaws on the current plan and selects the most promising flaw to solve according to certain evaluation criteria. The actual behavior of the solving procedure is determined by the specific configuration of an EPSL-based solver in terms of the particular strategy and the particular evaluation criteria applied during the search. In this context, the EPSL modular architecture allows to find the planner configuration which best meets the specific features of the problem to address. In particular, this work presents a modeling and solving approach which allows to realize a hierarchical reasoning with timelines [Umbrico et al., 2015].

5.1 The Modeling Language

EPSL takes inspiration from the APSI-TRF representation functionalities and uses the Domain Description Language (DDL) which is the modeling language that also APSI-TRF uses to model planning domains. DDL is a structured modeling language introduced in [Cesta and Oddi, 1996]. It provides the syntactic elements needed to describe timeline-based domains, i.e. state variables and synchronization rules. In addition, the Problem Description Language (PDL) is a language dedicated to describe problem instances.

Thus, an EPSL-based planner takes as input, a DDL and a PDL files representing respectively the description of a timeline-based domain and the description of a particular problem to solve. Given such input, an EPSL-based planner generates (if possible) a valid solution plan as output. In particular, EPSL relies on an extended syntax of DDL in order to comply with the formal characterization of timelines introduced in [Cialdea Mayer et al., 2016]. Specifically, EPSL introduces the syntactic constructs that allow users to specify controllability properties of state variable values (i.e. whether a value is controllable or uncontrollable) and the type of modeled state variables (i.e. whether a state variable is planned or external). Thus, the following sections provide a detailed description of the extended DDL/PDL language by exploiting the Rover planning domain introduced in Chapter 4.

5.1.1 The Domain Description Language

The Domain Description Language (DDL) is the language EPSL uses for domain modeling. The DDL provides the syntactic constructs needed to describe state variables, synchronization rules and all the information needed to characterize a timeline-based planning domain. This section introduces the "extended" DDL syntax by describing a timeline-based model designed for the Rover planning domain. In general, a DDL model is composed by the following parts: (i) general declaration; (ii) state variable specification; (iii) component specification; (iv) synchronization specification.

The code below shows the general domain declaration for the Rover planning domain. It declares the name of the planning domain, the temporal horizon and the types of parameters that the values of the components may assume. Specifically, the location parameter models the set of physical locations the rover can move to. In this specific domain the possible locations are discretized and modeled as symbols rather than as coordinates on a two-dimensional or three-dimensional space. Thus, location is an enumeration parameter whose values are included in a discrete set of symbols. The file parameter models the data files that the rover can communicate to share scientific information. The parameter is modeled as a numeric parameter whose values range within the interval [0, 100].

{highlights} ** DOMAIN** Rover { ** TEMPORAL_MODULE** tm = [0, 100];

** PAR_TYPE** * EnumerationParameter* location = { home, location1, location2, location3, location4 }

** PAR_TYPE** * NumericParameter* file = [0, 100];

... }

The state variable specification of a DDL model describes the features of the planning domain that must be controlled over time and their possible temporal evolutions. Considering the Rover example, the domain specification must model the navigation facility that allows the rover to move, the communication facility that allows the rover to communicate scientific data and the instrument that allows the rover to take samples to be analyzed. In addition, the domain specification must model the available communication windows during which the rover may actually send data.

The following code shows the state variable declaration modeling the functional and abstract behavior of the whole system to control, i.e. the planetary exploration rover. The RoverType state variable models the high-level tasks or states the rover may perform or assume over time. The value Idle() represents the fact that the rover is not operating and therefore it can receive goals, i.e. tasks to be performed. The value TakeSample(?location, ?file) represents a mission goal which asks the rover to take a sample at a specific ?location, perform some analysis on the gathered samples and communicate data (i.e. ?file) to the ground station. Both values are controllable and the related duration constraints do not bound them to some intervals. Consequently, the planner can dynamically decide the actual duration of these values according to the specific needs of the generated plans.

{highlights} ** COMP_TYPE** ** StateVariable** RoverType ( Idle(), TakeSample(location, file)) { ** VALUE** Idle() ** [1, +INF]** ** MEETS** { TakeSample(?location, ?file); }

** VALUE** TakeSample(?location, ?file) ** [1, +INF]** ** MEETS** { Idle(); } }

The code below shows the state variable specification for the navigation facility of the rover. The NavigationType state variable models the states and actions that the navigation module of the rover can assume or perform over time. The value At(?location) models the fact that the rover is still at a known ?location. This value is controllable and represents a temporally stable state because the related duration constraint does not specify a bound for the value. The transition function requires that a value GoingTo(?destination) follows a value At(?location) and that the parameter constraint ?location != ?destination holds. The parameter constraint guarantees that the rover does not move in case that the current location coincides with the desired destination.

The value GoingTo(?location) models the moving action of the rover. In this case the value is uncontrollable and a duration bound is specified [5, 11]. Thus, the system knows the estimated duration of the action, but it cannot completely control it. Namely, the rover can decide when to start moving towards a destination, but it cannot predict the time needed to reach the destination. Indeed, external factors, such as obstacles or the features of the ground, may affect the speed of the rover and therefore the time the rover takes to complete the GoingTo(?location) activity/task. Moreover, the related transition constraint requires that a value At(?destination) follows a value GoingTo(?location) and that the parameter constraint ?destination = ?location holds. Namely, the the rover is actually located at ?location (i.e. the desired destination) after the successful execution of theGoingTo(?location).

{highlights} ** COMP_TYPE** ** StateVariable** NavigationType ( At(location), GoingTo(location)) { ** VALUE** At(?location) ** [1, +INF]** ** MEETS** { GoingTo(?destination); ?location != ?destination; }

** VALUE** ** uncontrollable** GoingTo(?location) ** [5, 11]** ** MEETS** { At(?destination); ?destination = ?location; } }

The next block of code describes the state variable modeling the instrument payload of the rover. The InstrumentType state variable models the position the instrument assumes and the operations it may perform over time. It is a planned variable with both controllable and uncontrollable values. The values Unstowed() and Stowed() are temporally stable values that model respectively the idle state of the instrument and the operating state of the instrument. Namely, the instrument is not operative and cannot be used to take samples during Stowed(). Conversely, the instrument is operative and ready to take samples during Unstowed() . The transitions between the two states are modeled through values Unstowing() and Stowing(). They are both controllable values and their durations are fixed. Thus the system knows exactly how long the instrument takes to change from an idle state to an operative state and vice versa.

After activation through unstowing, the instrument must be placed over a target in order to take samples. Thus, the value Placing(?location) similarly to the GoingTo(?location) value, models the transition between the Unstowed and the Placed(?location) values. However, the Placed(?location), differently from the GoingTo(?location) value, is modeled as a controllable value with flexible duration. Indeed, the instrument is supposed to move among the reachable position without finding obstacles or any sort of external elements that may slow-down or even prevent the motion. Thus, it can be modeled as a controllable process whose flexible duration is determined by the minimum and maximum time needed by the instrument to reach the possible targets.

The value Placed(?location) models the fact that the instrument has been actually placed on a target and therefore it is ready to perform any operation on it. The Sampling value models the operation which allows the instrument to take samples of the target it is placed on. Such operation is uncontrollable because the time the instrument takes to take samples is affected by the particular shape and size of the target. This is a source of uncertainty of the environment and therefore the sampling operation is modeled as an uncontrollable process with an estimated lower and upper bounds for the duration.

{highlights} ** COMP_TYPE** ** StateVariable** InstrumentType ( Unstowed(), Stowing(), Stowed(), Unstowing(), Placing(location), Placec(location), Sampling(location)) { ** VALUE** Unstowed() ** [1, +INF]** ** MEETS** { Stowing(); Placing(?location); }

** VALUE** Stowing() ** [3, 3]** ** MEETS** { Stowed(); }

** VALUE** Stowed() ** [1, +INF]** ** MEETS** { Unstowing(); }

** VALUE** Unstowing() ** [3, 3]** ** MEETS** { Unstowed(); }

** VALUE** Placing(?location) ** [3, 7]** ** MEETS** { Placed(?target); ?target = ?location; }

** VALUE** Placed(?location) ** [1, +INF]** ** MEETS** { Sampling(?target); ?target = ?location; Placing(?newTarget); ?newTarget != ?target; Unstowed(); }

** VALUE** ** uncontrollable** Sampling(?target) ** [5, 18]** ** MEETS** { Placed(?location); ?location = ?target; } }

The communication facility of the rover is modeled by means of the CommType state variable. As the code below shows, the variable is composed by two values only. The Idle value models the fact that the communication facility is available for communicating data. The SendData value models the fact that the communication facility is actually sending data and cannot be used for other operation until the data transfer is complete. The communication task is modeled as an uncontrollable process whose actual duration is affected by external factors like the quality of the communication signal available and the size of the amount of data to be transferred. Thus, the time the rover takes to send data cannot be decided and therefore the related value is uncontrollable.

{highlights} ** COMP_TYPE** ** StateVariable** CommType ( Idle(), SendData(file)) { ** VALUE** Idle() ** [1, +INF]** ** MEETS** { SendData(?file); }

** VALUE** ** uncontrollable** SendData(?file) ** [11, 32]** ** MEETS** { Idle(); } }

Finally, the availability of the communication channel during the mission of the rover is modeled by means of the WindowType external state variable. All the values of an external variable are uncontrollable by definition and therefore the uncontrollable tag is not needed. The Available value models the fact that the communication channel is supposed to be available and data can be actually transferred during the related (flexible) temporal interval. Conversely, the NotAvailable value models the fact that the communication channel is supposed to be not available and no data can be transferred during the related (flexible) temporal interval.

As formally described in Chapter 4, external variables model features of the environment that are completely outside the control of the system. However, these features are relevant from the control and planning perspectives because their behaviors may directly or indirectly affect the behavior of the system. In this specific case, the availability of the communication channel affects the scheduling of the communication tasks of the rover. Clearly, the rover can neither decide or make hypothesis on the availability of the signal. Thus, according to Section 4.4, the behaviors of these features must be known and provided with the observations of the problem specification.

{highlights} ** COMP_TYPE** ** StateVariable** ** external** WindowType ( Available(), NotAvailable()) { ** VALUE** Available() ** [1, +INF]** ** MEETS** { NotAvailable(); }

** VALUE** NotAvailable() ** [1, +INF]** ** MEETS** { Available(); } }

After state variable declaration, the component specification of a DDL model aims at declaring the set of state variable instances that constitute the planning domain. The components of the DDL represent the instantiated data structure the planning system actually deals with in order to find plans. In this case, the DDL specification is composed by a component for each type of state state variable defined. The RoverController component represents an instance of the RoverType state variable. The Navigation component represents an instance of the NavigationType state variable. The Instrument component represents an instance of the InstrumentType state variable. The Communication component represents an instance of the CommType state variable. Finally, the Channel component represents an instance of the WindowType external state variable.

{highlights} ** COMPONENT** RoverController : RoverType; ** COMPONENT** Navigation : NavigationType; ** COMPONENT** Instrument : InstrumentType; ** COMPONENT** Communication : CommType; ** COMPONENT** Channel : WindowType;

The last part of a DDL planning model concerns the synchronization rule specification. While the state variable specification constrains the behaviors of the single features of the domain, synchronization rules specify global constraints aiming at coordinating domain components in order to realize complex behaviors of the system. The key point of the timeline-based modeling is that synchronization rules, differently from actions of classical planning, do not explicitly consider causal relationships among tokens of the timelines. Such rules "simply" specify additional constraints on the behaviors of state variables the planner must take into account when building the related timelines. The following code shows the synchronization rules defined for the Rover domain. These rules model the operational constraints introduced in Section 4.1 that allow the rover to successfully complete the mission.

The rule defined on the value TakeSample of component RoverController models the operational requirements the rover must follow to complete a mission goal. Specifically, the rule requires the rover to be located at the target position when performing sampling operations. Then, when the TakeSample task is completed the rover must send resulting data through the communication facility. According to these rules, the following temporal constraints must hold in the generated plans:

[TABLE]

Similarly, the synchronization rule on the value SendData of the component Communication models the operational requirements the rover must follow to successfully send data to the satellite. Specifically, the rule requires the rover to be still for the entire duration of the communication which in turn must be performed when the channel is available. Thus, the planner must generate plans that satisfy the following temporal constraints:

[TABLE]

Finally, in addition to operational requirements, sycnhronization rules may also model safety constraints. Namely, constraints needed to guarantee the safety of the system but not essential for the mission. The synchronization rule on value GoingTo of the Navigation component represents an example of such constraints. The rule requires the rover to keep the instrument stowed while moving in order to avoid collisions and preserve the safety of the device. Thus, every time the rover moves between two locations, following temporal constraint must hold:

[TABLE]

{highlights} ** SYNCHRONIZE** RoverController { ** VALUE** TakeSample(?target, ?file) { cd0 Navigation.At(?location); cd1 Instrument.Sampling(?target1); cd2 Communication.SendData(?file2);

  ** DURING [0, +INF] [0, +INF]** cd0;
  ** CONTAINS [0, +INF] [0, +INF]** cd1;
  ** BEFORE [0, +INF]** cd2;

  ?target1 = ?target;
  ?file2 = ?file;

} }

** SYNCHRONIZE** Communication { ** VALUE** SendData(?file) { cd0 Channel.Available(); cd1 Navigation.At(?location);

  ** DURING [0, +INF] [0, +INF]** cd0;
  ** DURING [0, +INF] [0, +INF]** cd1;

} }

** SYNCHRONIZE** Navigation { ** VALUE** GoingTo(?destination) { cd0 Instrument.Stowed();

  ** DURING [0, +INF] [0, +INF]** cd0;

} }

5.1.2 The Problem Description Language

The Problem Description Language (PDL) is the language EPSL uses for problem modeling. Given a domain specification, the PDL provides the syntax constructs needed to specify known facts about the world, the observations concerning the external variables of the domain (if any) and planning goals.

The following block of code represents a part of problem specification which declares the name of the problem instance, the planning domain it relies on and a set of known facts. In timeline-based planning, facts represent a partial description of the timelines of the domain components. Namely, they partially constrain the temporal behaviors of components by specifying a set of values the related variables can assume within known temporal bounds. Thus, the planning system builds the temporal behaviors of domain components by taking into account also the related known facts of the PDL.

These facts represent tokens on domain timelines and characterize the initial (partial) plan of the planning process. Namely, such tokens (partially) constrain the temporal behaviors of domain components and the solution plan must be built accordingly. For example, the token f0 in the block of code below specifies the starting position (i.e. the home location) of rover mission. Similarly, facts f1 and f2 specify respectively that the instrument and the communication facility of the rover are stowed and in idle state when the mission starts.

{highlights} ** PROBLEM** Rover_1task (** DOMAIN** Rover) { f0 ** fact** Navigation.At(?startLocation) ** AT [0, 0] [1, +INF] [1, +INF]; f1 ** fact Instrument.Stowed() ** AT [0, 0] [1, +INF] [1, +INF]; f2 ** fact Communication.Idle() ** AT [0, 0] [1,+INF] [1,+INF]**;

...

?startLocation = home; }

If a planning domain contains external variables the PDL must specify the related observations. According to Definition 4.4.23, observations must completely describe the temporal behaviors of external variables. Namely, they must specify the complete sequence of tokens that compose the timelines. The code below shows an example of observations for the external variables of the Rover planning domain. Specifically, the observations describe the sequence of (flexible) tokens that compose the timeline of the Channel component which models the availability of the communication signal during the mission.

{highlights} o1 ** fact** Channel.NotAvailable() ** AT [0, 0] [25, 30] [25, 30]; o2 ** fact Channel.Available() ** AT [25, 30] [80, 85] [55, 60]; o3 ** fact Channel.NotAvailable() ** AT [80, 85] [100, 100] [15, 20]**;

Finally, the PDL file contains the goal specification that, similarly to facts, represent a set of constraints concerning the temporal behaviors of domain components. Given such constraints (i.e. facts and goals), the planning system must build valid temporal behaviors (i.e. timelines) in order to complete a mission. Namely, the solving process is triggered by planning goals that represent requirements that solution plans must satisfy. The code below represents an example of a planning goal for the Rover domain. In this example, the goal g0 requires the rover to perform a TakeSample task within some temporal bounds.

{highlights} g0 ** goal** Rover.TakeSample(?tl, ?f) ** AT [0, 35] [22, 65] [1, 45]**;

?tl = location5; ?f = 1;

5.2 Architectural Overview

EPSL defines a flexible software framework to support the design and development of timeline-based applications. It is organized according to the Multilayered architectural pattern111A multi-layered architecture is a software architecture that uses many layers for allocating the different responsibilities of a software product [Buschmann et al., 1996]. The framework addresses the design and development of a timeline-based application from different perspectives ranging from the temporal reasoning mechanism to the definition of search strategies and heuristics. Each layer provides a set of ready-to-use algorithms and data structures that can be combined together to develop new planning instances. The modularity of the architecture allows users to easily integrate new features and improve the reasoning and representation capabilities of the framework. Figure 5.1 shows the main architectural elements that compose the EPSL framework. In general the system is composed of two macro layers, (i) the Representation layer and (ii) the Deliberative layer.

5.2.1 Representation Framework

The Representation layer is responsible for encapsulating and providing all the information and functionalities the solver needs to build timeline-based plans. The Temporal DataBase provides temporal reasoning mechanisms for checking the temporal consistency and inferring additional knowledge about the temporal relations of the plan. Similarly, the Parameter DataBase provides CSP-based reasoning mechanism for managing variable declaration and constraint propagation.

On top of these mechanisms resolvers and components provide a set of ready-to-use data structures and algorithms. These elements encapsulate the functionalities needed to manage timelines and timeline-based plans. In particular resolvers are the basic architectural elements for building timeline-based plans. They encapsulate the logic for managing plan flaws during the planning process. A flaw represents a particular issue concerning the plan which must be solved in order to find a solution. There are two types of flaws that must be managed: (i) planning goals represent flaws affecting the completion of the plan; (ii) threats represent flaws affecting the validity of the plan. Thus, each resolver is a dedicated algorithm responsible for detecting and solving a specific type of flaw.

Domain Components represent data structures modeling the different types of features of a planning domain. Specifically, each component aggregates a set of resolvers that determine the resulting behavior in terms of possible flaws that may concern the particular feature of the domain. Namely, the set of resolvers related to a component determine the conditions that must be solved in order to build valid temporal behaviors of the related feature (i.e. the timelines). Given this structure, the representation capability of the EPSL framework can be "easily" extended by introducing new domain components and new resolvers for building the related temporal behaviors. The higher the number of types of resolvers and components available the more the expressivity of the framework.

All the functionalities and information of the Representation layer are made available through the PlanDataBase interface which is a compound element encapsulating the complexity of plan management. Figure 5.2 provides a more detailed representation of the elements that compose the plan database and their relationships.

DomainComponents model the different types of feature the planning system can reason about. The PlanDataBaseComponent is a particular type of component which encapsulates other components of the domain (see the Composite design pattern [Gamma et al., 1995]) and provides access to the information of the plan through the PlanDataBase interface. The PlanDataBaseComponent uses the PlanRefinement resolver which is responsible for managing planning goals during the solving process. Specifically, the resolver provides functionalities for managing the expansion and/or unification of goals during plan refinement. Goal expansion refines the plan by creating and adding new tokens to timelines and applies the related synchronization rules that decompose the goal into a subset of sub-goals. Goal unification refines the plan by "merging" goals with already existing and compatible tokens on the timelines. Thus, unification does not require goal decomposition and the consequent generation of sub-goals.

StateVariables represent the basic data structures modeling the features of a planning domain. EPSL complies with the formalization described in Chapter 4. Therefore, state variables encapsulate information concerning the values the related feature may assume over time, the allowed transitions, their flexible durations and controllability properties. There are three types of state variables available: (i) ExternalStateVariable; (ii) FunctionalStateVariable; (iii) PrimitiveStateVariable. ExternalStateVariables model features of the domain that are completely uncontrollable. They use the ObservationChecker resolver which is responsible for verifying the observations provided as input through the problem specification. The resolver verifies that the observations represent a complete and valid temporal behavior satisfying the domain constraints of the related external variable (i.e. the value transition function).

FunctionalStateVariables and PrimitiveStateVariables are both planned variables. The former type of state variable models complex tasks/activities of the problem that must be further decomposed through synchronization rules. The latter type of state variable models tasks/activities that can be directly executed by the system. These two types of state variable use StateVariableGapSolver and StateVariableScheduling resolvers to build timelines. StateVariableScheduling resolvers are responsible for handling scheduling threats of the plan in order to avoid temporally overlapping tokens on the timelines. StateVariableGapSolver resolvers are responsible for handling gap threats of the plan in order to avoid "empty" temporal intervals on the timelines. Finally, the BehaviorChecker resolver is responsible for checking the resulting temporal behaviors with respect to the value transition function of the related state variable specification.

5.2.2 Problem Solving

The Deliberative layer in Figure 5.1 deals with problem resolution. It relies on the representation functionalities of the Representation layer and encapsulates the logic for solving timeline-based problems. In particular, it provides a set of ready-to-use search strategies, heuristics and solving algorithms that can be composed in order to define new planning instances. Indeed, the key point of EPSL flexibility is the interpretation of a planner as a “modular” solver which carries out the reasoning process by combining several elements.

The Solver encapsulates the particular structure of the reasoning process. Broadly speaking, the reasoning process is a general plan refinement algorithm which is supported by a Strategy and Heuristics encapsulating some criteria to guide the search. The former provides criteria for managing the fringe of the search space and selecting the "best" (partial)plan to expand next. The latter encapsulates criteria for managing the flaws of the plan and selecting the most promising flaws to solve. Figure 5.3 shows the main architectural elements that compose an EPSL-based planner and their relationships.

Similarly to classical planning, the planning process can be summarized as the search of a solution plan with some desired features on a space of possible plans. The search tree is composed by a set of search nodes that encapsulates a particular plan representing a possible status of the system in terms of temporal behaviors. The fringe of the search tree consists of the subset of nodes not visited yet. The SearchStrategy is responsible for managing the fringe of the search space by encapsulating some particular criteria for assessing the nodes composing the fringe. The planning process finds the "best" solution according to the particular search strategy used. The EPSL framework provides the user with two search strategies. The DepthFirst strategy (DFS) realizes a blind search where the planning process is simply guided towards the last-generated nodes of the search (i.e. the deepest ones in the search space). The MakespanOptimization strategy analyzes the temporal information of different pans in order to find the solution plan with the minimum makespan. Namely, this strategy tries to generates the most temporal efficient plan possible.

The FlawSelectionHeuristic encapsulates the logic for managing and assessing flaws of a plan detected during the refinement process. Specifically, a FlawSelectionHeuristic relies on a set of FlawFilters that select the "most relevant" flaws the planning process must consider for plan refinement. Each FlawFilter encapsulates a particular criterion for assessing flaws of a plan. The TypeFilter and the FailFirstFilter represent two straightforward selection criteria that take into account information concerning the particular flaws detected. The former filters flaws according to their type. The filter structures the planning process by requiring to address all planning goals first, than, scheduling threats and lastly, gap threats. The latter encapsulates the fail-first principle of constraint programming according to which the flaws with the minimum number of available solutions are selected first (i.e. the hardest flaws to solve). HierarchyFilter and SemanticFilter represents more complex selection criteria that will be explained in more detail in the next section. Broadly speaking, these two types of filters want to provide selection criteria that make decisions according to relationships and features that can be extracted by analzying the domain specification.

5.2.3 The General Solving Procedure

The Solver element of Figure 5.3 manages the structure of the planning process by "coordinating" the particular strategy and the particular heuristics used to search solutions. In general, EPSL-based planners follow a general plan refinement search procedures where an initial plan is iteratively refined by solving flaws until a solution plan is found. There are two available implementations the EPSL framework provides. The BestFirst solver represents a simple implementation of the planning process, which returns the "best" solution found according to the particular search strategy adopted. The PseudoControllabilityAware extends the behavior of the "best-first" solver by adding the assessment of the pseudo-controllability property of the plan during the planning process. Specifically, the PseudoControllabilityAware solver returns a pseudo-controllable plan if possible, or the "best" non-pseudo-controllable plan (if pseudo-controllability cannot be satisfied). Before entering into the details of the motivations for pseudo-controllable plans and its implications with respect to the planning process, this section provides a detailed description of the best-first solving procedure of an EPSL-based planner.

Algorithm 1 describes the abstract solving procedure of an EPSL-based planner. Basically, the reasoning process performs a plan refinement procedure which iteratively refines the current plan $\pi$ by detecting and solving flaws. EPSL instantiates the planner solving process over the tuple $\langle\mathcal{P},\mathcal{S},\mathcal{H}\rangle$ where $\mathcal{P}$ is the specification of a timeline-based problem to solve, $\mathcal{S}$ is the search strategy the planner uses to expand the search space, and $\mathcal{H}$ is the heuristic function the planner uses to select the most promising flaw to be solved.

The plan database $\pi$ is initialized on the problem description $\mathcal{P}$ (row 2) and the procedure iteratively refines the plan until a solution or a failure is detected (rows 6-32). At each iteration (rows 8-25) the procedure analyzes the current plan database $\pi$ by detecting flaws that must be solved $\phi^{0}(\pi)$ (row 9). Then the set of detected flaws $\Phi^{0}$ is filtered by applying the selected heuristic function $\mathcal{H}$ and the subset of equivalent flaws is extracted $\Phi^{*}\subseteq\ \phi^{0}$ (row 11). Each flaw $\phi^{*}_{i}\in\ \Phi^{*}$ may have one or more solution $N_{\phi^{*}_{i}}=\{n_{1},...,n_{t}\}$ (row 15). If no solution is found for a particular flaw $|N_{\phi^{*}_{i}}|=0$ then the flaw is unsolvable and backtrack is needed (rows 17-19). Otherwise each available solution represents a branch of the search and is added to the fringe (rows 21-24). The iteration ends by selecting a node from the fringe and refining the plan $\pi$ accordingly (row 31). The search goes on until a plan with no flaws is found, i.e. a solution plan (row 7).

Flaw Filtering

In general, flaw selection is not a backtracking point of the search but it can strongly affect the performance of the planning process. From the search point of view, each solution of a flaw determines a branch of the search tree. Thus a "good" selection of the next flaw to solve can prune the search space by cutting off branches that would lead to unnecessary or redundant refinements of the plan. Considering a particular branch of the search tree, a FlawSelectionHeueristic determines an "ordering" in flaw resolution the planner must follow in order to reduce the branching factor and avoid an exhaustive expansion of the search tree. The EPSL framework provides FlawSelectionHeuristics in shape of a pipeline of FlawFilters which is structured as follows:

$\phi^{0}...\xrightarrow{f_{i}(\pi,\phi^{i-1})}\phi^{i}\xrightarrow{f_{i+1}(\pi,\phi^{i})}\phi^{i+1}...\xrightarrow{f_{k}(\pi,\phi^{k-1})}\phi^{*}\subseteq\ \phi^{0}$

The pipeline represents a quite flexible structure which can be easily adapted or extended by taking into account different types of filters and also different combinations of the same set of filters. Each specific configuration of the pipeline results in a different behavior of the solving process. Given a partial plan $\pi$ with an initial set of flaws $\phi^{0}$, a sequence of filters $f_{i}$, with $i=1,...,k$, is applied in order to extract the subset of relevant flaws to consider for plan refinement, $\phi^{*}\subseteq\ \phi^{0}$.

The flaws composing the last set represent equivalent choices from the heuristic point of view, therefore they are all taken into account for plan refinement. The amount of information a particular heuristics function provides to the search can be estimated by checking the number of flaws actually filtered with respect to the initial set. If the set of flaws obtained by the application of a heuristic function $h(\pi)$ is equal to the initial set, i.e. $\phi^{*}=\phi^{0}$, then it is possible to argue that the heuristic function $h(\pi)$ is not informed. Indeed, in such a case the heuristics does not provide any useful information to problem resolution, therefore the resulting behavior of the solving process is a blind search.

5.3 Looking for Pseudo-controllable Plans

As discussed previously, EPSL relies on the formal characterization of the temporal uncertainty and controllability problem with respect to timelines given in [Cialdea Mayer et al., 2016]. Therefore the framework can represent and reason about the temporal uncertainty of the planning domain by taking into account the uncontrollable values and external features in order to generate plans with some desired properties concerning their execution. In general, the execution of a plan is a complex and hard task which requires the system to actually interact with the environment. There are many factors that may affect the executive process and even prevent the complete execution of the plan. The system must be able to handle the observations concerning the uncontrollable dynamics of the environment in order to dynamically adapt the plan as needed and complete the execution.

Observations allow the system to check whether the execution is diverging from the expected plan or not. If the execution is diverging, then the system must manage the plan and react to exogenous events accordingly. In the best case observations comply with the expected plan and no change is needed. Sometimes instead, it may happen that the system must dynamically adapt the ongoing plan to the observations in order to proceed with the execution (e.g. a delay of the execution of an uncontrollable activity which propagates to the not executed portion of the plan). In the worst case, the observations and the plan are incompatible and replanning is required. It means that the execution is stopped in order to produce a new plan starting from the current situation. Once the deliberative process has generated the plan, the execution can start over again.

In this regard, a robust executive system must cope with the uncertainty of the environment and complete the process by adapting the plan to any expected exogenous event and perform replanning only if needed. There are many works in the literature that take into account temporal uncertainty and study the controllability property of a plan [Vidal and Fargier, 1999, Morris et al., 2001, Cesta et al., 2010, Cialdea Mayer and Orlandini, 2015, Nilsson et al., 2016]. Specifically three types of controllability properties have been defined: (i) weak controllability; (ii) strong controllability; (iii) dynamic controllability.

Dynamic controllability is the most relevant property with respect to planning and execution in the real world. Broadly speaking, dynamic controllability concerns with the capability of an executive system to find a valid execution strategy which takes feasible dispatching decisions (i.e. it schedules the start of plan’s activities) by reasoning only on the past history and the received observations. It is not easy to deal with these properties during the planning process. They are usually checked with a post-processing step after the planning phase and before starting the execution of the plan. With respect to planning, an interesting property worth to be considered, is the pseudo-controllability property. Indeed, the pseudo-controllability property of a plan represents a necessary but not sufficient condition for its dynamic controllability [Morris et al., 2001].

The pseudo-controllability property of a plan aims at verifying that the planning process does not make hypotheses on the actual duration of the related uncontrollable activities. Specifically, pseudo-controllability verifies that the planning process does not reduce the duration of uncontrollable values of the domain during plan generation. Thus, a timeline-based plan is pseudo-controllable if all the flexible durations of uncontrollable tokens composing the timelines have not been changed with respect to the domain specification. Although, pseudo-controllability does not convey enough information to assert the dynamic controllability of a plan, it represents a useful property that can be exploited for validating the planning domain with respect to temporal uncertainty. Indeed, if the planner cannot generate pseudo-controllable plans, then the planner cannot generate dynamically controllable plans.

The PseudoControllabilityAware solver of the EPSL framework (see Figure 5.3) is responsible for generating pseudo-controllable plans (if possible). In this way, EPSL realizes an planning framework capable of taking into account (part) of the controllability problem by generating pseudo-controllable plans that can be further investigated. Such an integration between planning and execution, envisages a unified framework which allows plan-based controllers to rely on a common representation of the problem. The deliberative and executive capabilities share the same formal representation of the plan enabling a more flexible and effective management of the control process. This objective represents an ongoing development for the EPSL framework which has been partially addressed already as Chapters 6 shows.

Algorithm 2 describes the "extended" solving procedure implemented by PseudoControllabilityAware solver. Similarly to Algorithm 1, the solving procedure is an iterative partial plan refinement which searches pseudo-controllable plans. If no pseudo-controllable plans can be found, the procedure tries to find a non pseudo-controllable plan before ending. Thus the procedure returns a failure if neither a pseudo-controllable plan nor a non pseudo-controllable plan have been found. Similarly to Algorithm 1, the procedure is instantiated on the tuple $\langle\mathcal{P},\mathcal{S},\mathcal{H}\rangle$ whose elements represent the problem description, the search strategy and the flaw selection heuristic respetively.

The plan database $\pi$ is initialized on the problem description $\mathcal{P}$ (row 3). The procedure manages two distinct fringes during the search (initialized at row 5 and row 6). The pseudo-controllable fringe $F_{pc}$ is the fringe used when searching for pseudo-controllable plans. The non pseudo-controllable fringe $F_{\neg pc}$ is the fringe used when no pseudo-controllable plans have been found by the search. The solving procedure iteratively refines the plan until a solution is found (rows 8-47). At each iteration, the solving process checks the current plan for pseudo-controllability by analyzing the temporal uncertainty features of the domain (rows 10-12), i.e. uncontrollable and external values. If the flexible duration of uncontrollable values has not been changed (row 12) then the procedure starts refining the current plan as in the regular procedure described previously in Algorithm 1. If the pseudo-controllability condition does not hold (row 32) then the current plan is placed into the non pseudo-controllable fringe $F_{\neg pc}$, the procedure skips the refinement of the plan and the search goes on by extracting the next plan from the fringe (rows 38-40). However, if the fringe is empty then no pseudo-controllable plans can be found. Consequently, the search goes on by extracting nodes from the non pseudo-controllable fringe $F_{\neg pc}$ (rows 35-37). In such a case, it means that if a solution exists, then the plan is not pseudo-controllable and therefore the plan cannot be dynamically controllable. If both the fringes are empty then the procedure returns a failure (row 42), otherwise the search will end with a solution plan which can be either pseudo-controllable or not.

5.4 Hierarchical Planning with Timelines

In general, the design of effective models capable of capturing all the relevant features of a particular system is an issue in plan-based controller development. Indeed, a model must capture all (and only) the information about the system and the working environment that are really relevant with respect to the objectives of a particular application. Given a particular planning technique, it is not easy to find the appropriate abstraction level and structure the model accordingly. Typically, structured models support the solving process by encoding domain-specific knowledge about the problem. In this regard, hierarchical approaches like HTN have been successfully applied especially in real-world scenarios. This section introduces a hierarchy-based methodology for the design of timeline-based applications which takes inspiration from HTN approaches. In addition, a domain independent heuristics capable of leveraging the resulting structure of planning domains is presented.

5.4.1 Hierarchical Modeling Approach

In plan-based control systems the focus is usually on controlling an autonomous agent in order to perform complex tasks in a specific working environment (e.g. an industrial robot in a manufacturing work-cell). An effective timeline-based model must capture all the features and the related operational constraints that are relevant with respect to the control problem. The timeline-based model must describe the available capabilities that allow the system to actually interact with the environment and perform operations (e.g. actuators, sensors, tools), the features of the environment that may affect the behavior of the system, as well as the resulting high-level tasks (i.e. complex activities) the system can perform. Thus, it is possible to organize the information a timeline-based model must capture in three different levels of abstraction:

• •

The functional level concerns the high level tasks the agent can perform. It characterizes the high-level goals the timeline-based system can plan for.

• •

The primitive level concerns the internal elements that compose the agent. It characterizes the capabilities of the system in terms of the low-level tasks, or commands the agent’s components can directly execute. Namely, the primitive level deals with the representation of devices and facilities the system is endowed with (e.g. actuators or sensors) that can be actually used to solve a problem.

• •

The external level concerns the environment the agent must care about. It is orthogonal with respect to other levels and characterizes the dynamics of the environment the agent must interact with. Namely external level concerns the element of the domain that are outside the control of the agent but whose behavior may affect the outcomes of the activities needed to solve a problem.

According to this organization, Figure 5.4 shows the general structure of a timeline-based domain. The functional and primitive levels are directly related each other. The external level instead is orthogonal to the others as it may have implications at both levels. Within this structure, the model must specify a hierarchical decomposition of high-level tasks in terms of relations between low-level tasks. Such relationships may require several decomposition levels according to the complexity of the considered domain. In any case, the hierarchical decomposition starts with a high-level task representing a planning goal, and ends with a set of primitive tasks that can be directly executed by the system. The arrows in Figure 5.4 represent such a decomposition. Some arrows (the red dotted ones) specify relations between functional (or even primitive) values and external values. In these cases, the arrows represent condition checking rather then decomposition. Namely they represent conditions that must hold in the plan with respect to some features of the environment, rather then activities to perform. Following the three abstraction levels, the envisaged hierarchical modeling approach identifies three types of state variables composing a timeline-based domain. They are the (i) the functional variables, (ii) the primitive variables and (iii) the external variables.

Functional variables provide a logical representation of the agent in terms of the high-level task the agent can perform, notwithstanding its internal composition. The values of this type of variables are controllable and represents the high-level planning goals the related timeline-based system can plan for. With regards to the Rover domain, the Rover planned state variable is a functional variable which models the high-level tasks the rover can perform. Specifically, it models the rover in terms of the TakeSample activities that represent the planning goals of the problem.

Primitive variables model a specific physical/logical component of the system. The values of this type of variables represent state/actions the related component of the system is actually able to assume/perform over time. These values may have bounded flexible durations and may be either controllable or not. If such a value is tagged as uncontrollable, it means that the system cannot decide the actual duration of the related activity during execution (specifically the system can decide when to start the activity but not when the activity ends). These are the values the planning process looks for in order to check if the pseudo-controllability property of the plan is satisfied. For example, the SendData value of Communication variable is tagged as uncontrollable because the actual duration of the communication activities is affected by factors that are not under the control of the rover (e.g. the size of the data file, or the quality of the communication channel).

External variables model the features of the environment that are completely uncontrollable and whose behaviors may affect the operations of the system. Namely such variables model conditions that must hold in order to successfully carry out the tasks of the system. These variables can only be observed and the related timelines are included into the description of the problem. Therefore, the planning system must adapt the plan to the particular observations received (again, without making any hypothesis on their actual durations in order to comply with pseudo-controllability property). With regards to the Rover planning domain, the generated plan must comply with the observations concerning the communication channel such that the SendData activity is performed when the channel is supposed to be available.

Task decomposition is realized by means of synchronization rules that, like methods in HTN, connect adjacent abstraction levels of the domain by specifying relationships between different tasks. Namely, synchronization rules describe a top-bottom task decomposition specifying how the high-level tasks (i.e. functional values) are implemented by the internal components of the system (i.e. the primitive values) and how their execution is related to the environment (i.e. external values).

5.4.2 Building the Dependency Graph

It is possible to observe that a synchronization rule basically represents a dependency between two or more variables of the domain. It means that variables affect the temporal behavior of other variables through synchronization rules and the related temporal constraints between tokens. Let us consider a synchronization rule $S_{v_{A,i}}$ which applies to value $i$ of a state variable $A$ ($v_{A,i}$) and contains a temporal constraint between $v_{A,i}$ and a value $j$ of a state variable $B$ ($v_{B,j}$). Such a temporal constraint affects the behavior of state variable $B$ and consequently the building process of the related timeline (i.e. the timeline of state variable $B$). Thus, the synchronization rule $S_{v_{A,i}}$ determines a dependency between state variable $A$ and state variable $B$. Namely, the tokens that compose timeline $A$ and their temporal allocation affect the token that compose timeline $B$ and their temporal allocation. According to this observation, it is possible to analyze the synchronization rules of a domain specification in order to build a Dependency Graph (DG) encoding the relationships between domain components. Specifically, a DG is a directed graph which provides a relaxed representation of the dependencies between the components of a planning domain.

Definition 5.4.31**.**

A dependency graph DG is a directed acyclic graph defined by the pair $\langle V,R_{d}\rangle$ where: (i) $V$ is the set of nodes of the graph representing the components of the planning domain; (ii) $R_{d}$ is the set of (directed) edges between nodes representing dependency relationships between domain components

Algorithm 3 describes the building procedure of the DG. The procedure takes as input the planning domain and initializes the dependency graph $G_{dg}$ on the set of state variables of the domain (row 3). The dependencies between components are generated by analyzing the synchronization rules of the domain (rows 6-24). For each synchronization rule $S_{v_{A,i}}$, where variable $v_{A,i}$ represents the triggerer of the synchronization ($A$ the component the value belongs to, $i$ the id of the value), the related temporal constraints are taken into account to compute dependencies (rows 8-21). For each temporal constraint $r_{k}$ the algorithm checks if the relation is reflexive (row 10). If a temporal constraint involves two values belonging to the same component (i.e. it represents a reflexive dependency relation) then the constraint is ignored. Otherwise a new dependency is added to the graph concerning the reference and the target components of the relations (rows 11-16). Every time a new edge is added, the graph is checked for cycles (row 18). If a cycle is detected, it is caused by the last added dependency which is discarded and removed from the graph (row 20). The procedure continues until all synchronization’s temporal constraints have been analyzed and the resulting dependency graph $G_{dg}$ is returned.

The DG the procedure generates, is acyclic for construction. Indeed, the procedure discards edges (i.e. temporal relations) that introduce cyclic dependencies into the graph. Thus, the DG relaxes the dependency relationships of the domain by considering only an acyclic subset of them. A DG is said to be complete if all the dependencies of the domain are modeled. Often, given the hierarchical modeling approach described in the previous section, the dependencies of the domain are acyclic. The resulting DG is complete and encodes the hierarchy of the planning domain which can be easily extracted by analyzing the graph (e.g. by means of a topological sort algorithm if the DG does not contain cycles).

Figure 5.5 shows a general example concerning the hierarchy extraction process from domain synchronizations. Each synchronization rule specifies temporal constraints that may involve values of different state variables. Temporal constraints of different synchronizations can be distinguished according to their color in Figure 5.5 (i.e. temporal constraints with the same color belong to the same synchronization rule). Each temporal constraint between values of different components $v_{A,i}\in\ C_{A}$ and $v_{B,j}\in\ C_{B}$ determines a dependency relation between $C_{A}$ (the source of the relation) and $C_{B}$ (the target of the relation). Thus such a constraint is encoded by an edge $r_{A,B}\in R_{d}$ of the DG, where $A,B\in V$.

Given the DG, it is possible to extract the hierarchy of the domain as shown in Figure 5.5. Indeed, an edge connecting a node $A$ to a node $B$ in the DG, implies that component $B$ depends on component $A$. Thus the component $A$ is higher than component $B$ with respect to the hierarchy. Otherwise, If there is not a direct path connecting a node $A$ to a node $B$ in the DG, then no implications can be made concerning their relationship. In such a case, the related domain components are supposed to be at the same level of the hierarchy. Such a hierarchy can guide the solving procedure to search for solutions. In this regard, the HierarchyFilter element of Figure 5.3 encapsulates a flaw selection criterion which selects flaws according to the hierarchical level of the component they belong to. The rationale of the selection criteria is that, given a set of flaws to solve, the "best" choice is to start solving flaws that belong to the most independent component of the domain. Solving dominant flaws of the plan may implicitly solve other secondary flaws of the plan or even prune the search space by removing redundant or unfeasible flaw solutions.

Chapter 6 Planning and Execution with Timelines under Uncertainty

Plan generation is only a part of the problem when controlling a complex system with plan-based technologies in AI. The execution of a plan is a complex process which can fail even if the plan is valid with respect to the domain specification. During execution, the system must interact with the environment, which is uncontrollable and therefore the execution of the activities can be affected by external factors. A robust executive system must cope with such exogenous events and dynamically adapt the plan accordingly during execution. In order to deploy timeline-based applications in real-world scenarios, the EPSL planning framework has been extended by introducing executive capabilities. The executive relies on the same semantics of timelines the planning process relies on. Thus, the executive leverages information about the temporal uncertainty of the problem in order to properly manage the execution of the plan. In this way, EPSL realizes a uniform software framework for planning and execution with timelines under uncertainty. This chapter provides a detailed description of the extended EPSL framework and the related approach to execution. Moreover, the chapter introduces a real-world manufacturing scenario for Human-Robot Collaboration (HRC) where EPSL and the related planning and execution capabilities have been successfully applied.

6.1 Model-based Control Architectures

The classical approach for building model-based controllers relies on the three-layered architecture described in [Gat, 1997]. These three layers are (starting from the bottom) the functional layer, the executive layer and the planning/scheduling layer. Traditional autonomous control architecture follow this structure and the most relevant works concern: IPEM [Ambros-Ingerson and Steel, 1988], CPEF [Myers, 1999], the LAAS architecture [Alami et al., 1998] which relies on the IxTeT-eXeC [Lemai and Ingrand, 2004], the Remote Agent Experiment [Jonsson et al., 2000b] and ASE [Chien et al., 1999]. Each layer usually requires different reasoning and representation technologies. The integration of such different technologies is usually an issue for developing this type of controllers. Often, the planning cycle is monolithic making scalability and fast reaction time another issue of this type of controllers.

Other approaches like CLARAty [Nesnas et al., 2008], try to overcome some of these drawbacks using an architecture with only two layers. A functional layer and a decision layer. The decision layer integrates planning and execution through a shared data structure (i.e. the plan database) synchronizing planning and execution data that rely on two different representations, i.e. CASPER [Knight et al., 2001] for planning and TDL [Simmons and Apfelbaum, 1998] for execution. CIRCA [Goldman et al., 2002] proposes an intelligent controller in hard real-time which leverages reactive planning to implement automatic controller synthesis.

IDEA [Muscettola et al., 2002, Aschwanden et al., 2006] was the first agent control architecture utilizing a collection of controllers, each interleaving planning and execution in a common framework. The main drawbacks of IDEA are the lack of a clear conflict-resolving policy between controllers and the lack of an efficient planning algorithm for integrating the current states of controllers. The Teleo-Reactive Executive (T-Rex) [Py et al., 2010] was designed to overcome these restrictions using a collection of controllers (called reactors) implemented as different instances of EUROPA planner [Barreiro et al., 2012]. The novelty of T-Rex was the capability of realizing a systematic infrastructure which defines the interactions among reactors.

6.2 Extending the EPSL framework with Execution

The executive system is responsible for managing the execution of timeline-based plans by iteratively sending commands to the system and receiving observations concerning the actual state of the environment. Thus, the executive must verify whether the perceived behavior of the world (i.e. the system and the environment) complies with the expected plan and must react accordingly in case of conflicts.

Figure 6.1 shows the extended architecture of the EPSL framework with the main architectural elements. The executive relies on the same representation functionalities the deliberative relies on. Therefore, the system can manage the execution of the activities of the plan according to their controllability properties. The obtained EPSL framework represents a unified tool capable of seamlessly dealing with planning and execution of timelines with uncertainty. Plan execution manages particular information representing states and conditions that must be monitored during the execution of timelines. The Executive Plan Database Manager encapsulates this kind of information by extending the functionalities of the Representation layer. Specifically, it manages information concerning the execution state of plan’s tokens and the related execution dependencies. The temporal constraints of a timeline-based plan entail dependencies determining whether a token can actually start/end execution or not. Given a timeline-based plan to execute, the Executive Plan Database Manager extracts execution dependencies dynamically and encodes this information into a dedicated data structure called Execution Dependency Graph (EDG).

Figure 6.2 shows the elements composing a general EPSL executive and their relationships. In particular, the figure shows the additional elements the executive needs to exchange information/signals with the specific environment and robotics platform. The MoveItConnector and the MoveItListener represent two elements used within the research project FourByThree described in Section 6.3. They encapsulate the complexity of the remote communication with the robot and provide the EPSL executive with a set of local execution services (see the Proxy design pattern [Gamma et al., 1995]). In particular, the MoveItConnector encapsulates the set of low-level commands the robot can execute according to the operational interface of system. The MoveItListener instead, allows the executive to receive asynchronous messages from the system as well as the environment concerning the results of execution requests (i.e. feedbacks).

6.2.1 The Execution Process

The execution process consists of control cycles whose frequency determines the reactivity of the executive and the advancement of time. Given the temporal horizon of the plan, the execution process discretizes the temporal axis by means of a number of temporal units, called ticks, according the needed frequency. Each control cycle of the process is associated with a tick and realizes the execution procedure. Broadly speaking, the execution procedure is responsible for detecting the actual behavior of the system (closed-loop architecture), for verifying if the system and also the environment behave as expected from the plan and for for starting the execution of the activities of the plan. The procedure is composed by two distinct phases, the synchronization phase and the dispatching phase. At each tick (i.e. control cycle) the synchronization phase manages the received execution feedbacks/signals in order to build the current status of the system and the environment. If the current status is valid with respect to the plan, then the dispatching phase decides the next activities to be executed. Otherwise, if the current status does not fit the plan, an execution failure is detected and replanning is needed. Indeed, the current plan does not represent the actual status of the system and the environment and therefore replanning allows the executive to continue the execution process with a new plan, which has been generated according to the observed status and the executed part of the original plan.

Algorithm 4 describes the general control procedure of the executive and its related sub-procedures. The procedure takes as input the plan $\Pi$ to be executed and the clock $\mathcal{C}$ which determines the frequency of the control cycles. First of all, the procedure analyzes the plan $\Pi$ in order to identify execution dependencies among tokens of the timelines. This information is encapsulated by a dedicated structure $\pi_{exec}$ (row 3) the procedure uses during execution. The procedure iteratively executes the plan until all the tokens have been executed (rows 5-12). The timing of the iterations of the procedure is determined by the clock $\mathcal{C}$. Indeed, the procedure waits a signal from $\mathcal{C}$ which communicates the current execution time $\tau$ (row 7). Then, the procedure checks the status of the execution by calling the Synchronize sub-procedure (row 9) and ends the execution cycle by calling the Dispatch sub-procedure (row 11).

Figure 6.3 shows the runtime behavior of an EPSL executive and its interactions with Clock, PlanMonitor and PlanDispatcher elements shown in Figure 6.2. The Executive manages the structure of the control cycle by coordinating the synchronization and dispatching steps. The Clock iteratively generates control events by sending onTick(tick) signals to the Executive, according to the desired frequency. The higher is the frequency of the clock the higher is the reactiveness of the control process. Clearly, the clock’s frequency must be compatible with the response time of the system. The PlanDispatcher and the PlanMonitor are the elements responsible for managing respectively the dispatching and synchronization steps within the control cycle. Thus, as Figure 6.3 shows, every time the Executive receives a signal from the Clock, it coordinates the PlanMonitor and the PlanDispatcher in order to complete the control cycle. Specifically, the Executive calls the PlanMonitor to handle the synchronization step and build the current pereceived state of the system and the environment (i.e. the current situation). Then, the Executive calls the PlanDispatcher to handle the dispatching step according to the current situation and the current execution time.

The Synchronization Phase

The synchronization phase monitors the execution of the plan by determining if some divergencies occur between the expected plan and the observed behavior of the system and the environment. Namely, at each iteration the synchronization phase builds the current situation by taking into account the current execution time, the expected plan and the feedbacks received during execution. Figure 6.4 shows the elements involved within the synchronization phase and their interactions. The PlanMonitor is responsible for propagating observations concerning the actual duration of the dispatched activities and detecting discrepancies between the real-world and the plan. The Executive receives feedbacks about the successful execution of dispatched commands or failure. The PlanMonitor manages these feedbacks in order to detect if the actual duration of tokens comply with the plan. If the feedbacks comply with the plan then, the status of the related tokens can change from in-execution to executed. Otherwise, an inconsistency is detected (i.e. the current situation does not fit the expected plan) and a failure is notified to the Executive which must react accordingly (e.g. by re-planning).

The Dispatching Phase

The dispatching phase manages the actual execution of the plan. Given the current situation and the current execution time, the dispatching step analyzes the plan $\pi_{exec}$ in order to find the tokens that can start execution and dispatches the related commands to the underlying system. Namely, the dispatching step allows the Executive to advance execution by deciding the next tokens to execute. Figure 6.5 shows the elements involved within the dispatching steps and their interactions. The PlanDispatcher is responsible for making dispatching decisions of plan’s tokens. For each token, the PlanDispatcher checks the related start condition by analyzing the token’s scheduled time and any dependency with other tokens of the plan. If the start condition holds, then the PlanDispathcher can decide to start executing the token (i.e. the dispatcher propagates the scheduled start time into the plan). After dispatching, the status of the involved token changes from waiting to in-execution.

6.2.2 Managing the Execution Dependency Graph

A valid timeline-based plan consists of a set of timelines whose tokens represent valued temporal intervals satisfying all the constraints of the domain specification. According to the formal characterization given in Chapter 4, each token of a timeline is described by a duration and an end time (interval) satisfying the constraints of the plan. However, this information is not sufficient to properly manage plan execution. Temporal relations entail dependencies among tokens of a plan the executive must take into account during execution. For example a temporal relation of the form A meets B, entails that execution of token B must start as soon as execution of token A ends. As described in Chapter 5, such dependencies are encoded by the underlying temporal network and the inferred temporal bounds of the related temporal intervals, but the executive must explicitly model these relationships in order to "validate" the plan during execution. Timeline tokens represent flexible intervals and therefore, each token is characterized by a start execution condition and an end execution condition that allow the executive to decide their actual start and end times.

The Executive Plan Database Manager of Figure 6.1, extends information of the plan data-base in order to properly manage execution of timelines by means of EDG. An EDG is a data structure encapsulating information about the execution dependencies, the executive process relies on to make decisions about the execution of tokens. An EDG is a directed graph built by analyzing temporal relationships of the plan being executed. The nodes of the graph represent the tokens of the timelines composing the plan. Each node is associated with the current execution status of the related token of the plan. The (directed) edges represent execution dependencies between nodes (i.e. tokens). The tokens of a plan represent flexible temporal intervals and therefore there are two types of edges modeling execution conditions. The start/end execution conditions model conditions that allow the executive to decide the actual start/end of a token during execution.

Definition 6.2.32**.**

An Execution Dependency Graph (EDG) is a directed graph representing execution dependencies among the tokens of a plan. An EDG can be formally defined as follows:

[TABLE]

where:

• •

$V$* is the set of nodes of the graph each of which encapsulates a token of the plan.*

• •

$E=E_{s}\cup\ E_{e}\subseteq\ \{\left(v_{i},v_{j}\right):v_{i},v_{j}\in\ V\land\ v_{i}\neq\ v_{j}\}$* is the set of edges of the graph representing execution dependencies between two (distinct) tokens of the timelines. The set $E$ is partitioned into two subsets: (i) $E_{s}$ contains the edges representing token start dependencies; (ii) $E_{e}$ contains the edges representing token end dependencies.*

• •

$\Gamma=\{waiting,starting,inexecution,executed\}$* is a set of constants representing the possible execution status the tokens of the plan may assume: (i) a token is in waiting status if the related start conditions are not satisfied and the executive must wait for its execution; (ii) a token is in starting status if the related start conditions are satisfied and the executive can actually start its execution (this status is particularly relevant for fully uncontrollable tokens as section 6.2.3 will describe); (iii) a token is in inexecution status if the executive is actually executing the token. It means that the executive has started the execution of the token (i.e. the executive has dispatched the start time of the token) but cannot end its execution because the related end conditions are not satisfied yet; (iv) a token is in executed status if its execution is complete. It means that, the end conditions of the token have been satisfied and the executive has ended its execution.*

• •

$\rho:V\to\Gamma$* is a status function mapping each node $v_{i}\in\ V$ (i.e. a token of the plan) to its current execution status $\rho(v_{i})=\gamma_{i}\in\Gamma$. For example $\rho(v_{i})=waiting$ means that the current status of the token related to $v_{i}\in\ V$, is $waiting$.*

• •

$\xi:E\to\Gamma$* is a dependency function mapping each edge of the graph (i.e. an execution dependency) to the required status of the destination node. Specifically, considering start execution conditions, given an edge $\left(v_{i},v_{j}\right)\in\ E_{s}$, the condition $\xi\left(v_{i},v_{j}\right)=executed$, means that the start condition of the node $v_{i}$ is satisfied if the status of node $v_{j}$ is $executed$. The executive can start the execution of the token related to node $v_{i}$ iff the execution of the token related to node $v_{j}$ is ended. The condition $\xi\left(v_{i},v_{j}\right)=inexecution$ means that the start condition of the node $v_{i}$ is satisfied if the status of node $v_{j}$ is $inexecution$. The executive can start the execution of the token related to node $v_{i}$ iff the executive is still executing the token related to node $v_{j}$. The condition $\xi\left(v_{i},v_{j}\right)=waiting$ means that the start condition of the node $v_{i}$ is satisfied if the status of node $v_{j}$ is $waiting$. The executive can start the execution of the token related to node $v_{i}$ iff the executive has not yet started the execution of the token related to node $v_{j}$. Finally, the condition $\xi\left(v_{i},v{j}\right)=starting$ means that the start condition of the node $v_{i}$ is satisfied if the status of node $v_{j}$ is $starting$. Analogous interpretations hold for end execution conditions represented by edges $\left(v_{i},v_{j}\right)\in\ E_{e}$.*

The EDG is built from the plan before starting execution. The executing conditions, are dynamically extracted from the timeline-based plan by analyzing the temporal relations. Specifically, the graph generation procedure encodes Allen’s temporal relations [Allen, 1983] in a set of start and end execution conditions between the involved tokens of the plan (i.e. nodes of the graph). For example, given A and B two tokens of a plan, the temporal relation A during B can be encoded into the EDG graph by adding two execution conditions. A start execution condition asserting that A can start execution iff B is "currently" in execution and, an end execution condition asserting that A can end execution iff B is "currently" in execution. These two conditions are encoded by two edges, both of which have the node related to the token A as the source, and the node related to the token B as the target.

Algorithm 7 describes the procedure building the EDG from the timeline-based plan to be executed. The procedure first initializes the graph (row 3). The nodes of the graph represent tokens of the plan with their current execution status. The procedure creates a new node for each token of the timelines (rows 5-10) and for each token of the external timelines (i.e. the observations) composing the plan (rows 12-17). The edges of the graph are generated by encoding the temporal relations of the plan (rows 19-29). Each temporal relation is "translated" into a set of start and end execution conditions the procedure adds to EDG as edges (rows 21-28). Each execution condition $\left(n_{h,i},n_{h,j},c_{h,lk}\right)$ represents an (directed) edge of the graph. The source node ($n_{h,i}$) represents the token the dependency relation refers to. The target node ($n_{h,j}$) represents the token the source of the relation depends on. The condition ($c_{h,k}$) represents the execution status of the target token enabling the execution of the source token. The procedure ends by returning a complete EDG.

6.2.3 Handling Uncertainty During Execution

An EDG encapsulates temporal dependencies between tokens of the timeline-based plans. However, the executive must also take into account controllability information concerning the values the tokens of the timelines are related to. There are different types of tokens the executive must deal with according to the controllability properties of the related value of the domain. Different types of tokens entail different execution policies and therefore, different state transitions that may be either controllable or not. Specifically, there are three types of tokens the executive must manage during execution. Figure 6.6 shows the state transitions of the controllable, partially-controllable and fully-uncontrollable tokens.

Controllable tokens whose state transitions are shown in Figure 6.6 (a), are completely under the control of the executive. The executive can decide the actual start time of the token and its duration. Thus, the state transition between waiting state and in-execution state, as well as the state transition between in-execution state and executed state are both controllable. In this case, the executive can actually dispatch the signals for starting/ending the execution of the related command.

Partially-controllable tokens whose state transitions are shown in Figure 6.6 (b), represent tokens the executive cannot completely control. The executive can decide the start time of this type of tokens and therefore the state transition between waiting state and in-execution state is controllable. However, the actual execution of this type of token is not controllable and therefore, the system can only observe the execution by waiting for a signal from the environment concerning the end of token execution (i.e. the executive cannot control the end of the execution of this type of tokens). When the end signal is received, the executive verifies the consistency of the plan with respect to the observation (i.e. the system checks if the end conditions and the schedule of the token comply with the observed behavior) and, the uncontrollable transition between in-execution state and executed state is triggered.

Fully-uncontrollable tokens whose state transitions are shown in Figure 6.6 (c), are completely outside the control of the executive. The executive may suppose when the token is about to start according to its schedule, but cannot decide its actual start time. Thus, a state transition between waiting state and starting state is controllable but it means that the executive is waiting for a signal from the environment which notifies the start of the execution of the token. The system can only observe the start of the execution and check if the signal (i.e. the exogenous event) received complies with the plan (i.e. the start execution dependencies and the schedule of the token are satisfied). When the executive receives the start signal from the environment, the uncontrollable state transition between starting state and in-execution state is triggered. Then, similarly to partially-controllable tokens, the executive waits the signal concerning the end of the execution of the token. When the signal is received, again the executive checks the consistency with respect to the plan and the related (end) execution dependencies and the uncontrollable state transition between the in-execution state and the executed state is triggered.

6.2.4 The Importance of Being (Temporally) Robust

Timeline-based plans are temporally flexible, hence associated with an envelope of possible execution traces. Temporal flexibility allows the executive to be less brittle during execution because the system is able to manage the temporal uncertainty of the activities of the plan. The flexible temporal intervals of the tokens composing the timelines of the plan allow the executive to "easily" absorb execution delays within the specified bounds.

However, it is not always possible to complete the execution without changing or adapting the plan. Temporal uncertainty and uncontrollability features of the environment may lead to uncontrollable behaviors the timeline-based plan is not able to "capture" and therefore, the control system is forced to generate a new plan according to the perceived situation in order to complete the execution. Indeed, the plan-based controller relies on a model which tries to describe the (flexible) behavior of the uncontrollable features of the domain. Uncontrollability may cause behaviors that do not comply with the plan. For example, the execution of an uncontrollable activity may last longer than expected from the model, or it can start later than expected from the plan. In such cases, the executive interrupts the current execution and starts a re-planning phase which tries to generate a new plan from the observed situation.

Re-planning takes into account the primitive variables of the domain for building the stable state (i.e. the problem specification) the planning process starts from in order to generate the new plan. The executive analyzes the timelines of primitive variables by setting the executed tokens with the related temporal information as facts of the problem specification. Also, the tokens generated from the observation that caused the execution failure are added to the facts of the problem together with their temporal information. Moreover, if the executive was executing some uncontrollable tokens when the failure was detected (e.g. the rover was moving between two locations), then (supposing the related activities are non-interruptible) the system can wait for the end of their execution and add the related facts to the problem specification. Given the resulting problem specification, a new plan is generated and the execution can continue starting from the point at which it was interrupted (i.e. execution failure).

Re-planning is needed because the execution of these plans is decided on the fly. Without an execution policy, a valid plan may fail due to wrong dispatching or environmental conditions (controllability problem [Vidal and Fargier, 1999]). It is possible to address this issue in a more robust way by leveraging recent research results exploiting formal methods to generate a plan controller suitable for the execution of a flexible temporal plan [Orlandini et al., 2013]. Namely, UPPAAL-TIGA, a model checker for Timed Game Automata (TGA), can be exploited to synthesize robust execution controllers of flexible temporal plans. A TGA-based method for the generation of flexible plan controllers can be integrated within the executive. In this case, the UPPAAL-TIGA engine is embedded within the planning and execution cycle generating plan controllers that guarantee a correct and robust execution. This is an important feature of the FourByThree Task Planner as it enforces a safe plan execution further enforcing that all the production requirements and human preferences are properly respected.

6.3 Human-Robot Collaboration: a Case Study

Human-Robot Collaboration (HRC) in manufacturing represents an interesting and quite complex application context which requires a tight interaction between a human operator and a robotic device (e.g. a robotic arm) to perform some factory operations. From the perspective of a plan-based control system, the envisaged environment is composed of two autonomous agents that share the same working environment and may operate independently or may collaborate by supporting each other. This type of application presents several challenges a plan-based control system must cope with in order to control the robot and guarantee a safe collaboration with the human. In general, there are three important features the control system must deal with in order to generate effective plans:

• •

Supervision, to represent and satisfy the production requirements needed to complete the factory processes.

• •

Coordination, to represent the activities the human operator and the robot must perform according to the Human-Robot Collaboration settings.

• •

Uncertainty, to manage the temporal uncertainty about the activities of the human operator that the system cannot control.

A key enabling feature is the capability to model and manage the temporal uncertainty concerning the behavior of the human operator. The human is an active "part" of the environment which is not under the control of the robot. The control system must take into account the (expected) behavior of the human in order to properly manage operations of the robot. Thus, HRC represents a relevant application context to leverage the feature of the timeline-based planning and execution framework described above. The following sections deal with the development of an EPSL-based dynamic task planing system within the FourByThree research project, for planning and execution in real-world HRC manufacturing scenarios.

6.3.1 The FourByThree Research Project

Industrial robots have demonstrated their capability to meet the needs of many application domains, offering accuracy, efficiency and flexibility of use. A relevant research challenge is the co-presence of robot and human in the same environment collaborating in a common goal. In general, when robot-worker collaboration is needed, there are a number of open issues to be taken into account, first of those is human safety that needs to be enforced in a comprehensive way. A key open trend in manufacturing is the design of shared fenceless working spaces in which safe human-robot collaboration is seamlessly implemented. The FourByThree research project111http://www.fourbythree.eu [etf, 2016] aims at designing, building and testing robust and configurable robotic solutions capable of collaborating safely and efficiently with human operators in industrial manufacturing companies. The overall aim of the project is to create a new generation of robotic solutions, based on innovative hardware and software, which present four main characteristics: modularity, safety, usability and efficiency. The envisaged robot services take into account the co-presence of three different actors: humans, robots and the environment.

A human-robot collaboration workcell is a bounded connected space with two agents located in it, a human and a robot system, and their associated equipment [Marvel et al., 2015]. The robot system consists of a robotic arm with its tools, its base and possibly additional support equipment. The workcell also includes the workpieces and any other tool associated with the targeted task and dedicated safeguards (physical barriers and sensors such as, e.g., monitoring video cameras). In such a working environment, four different degrees of interaction between a human operator and the robot can be defined [Helms et al., 2002]. In all these cases, it is assumed that the robot and the human may need to occupy the same spatial location and interact according to different modalities:

• •

Independent, the human and the robot operate on separate workpieces without collaboration, i.e. independently from each other;

• •

Synchronous, the human and the robot operate on sequential components of the same workpiece, i.e. one can start a task only after the other has completed a preceding task;

• •

Simultaneous, the human and the robot operate on separate tasks on the same workpieces at the same time;

• •

Supportive, the human and the robot cooperate to complete the processing of a single workpiece, i.e. they work simultaneously on the same task.

Different interaction modalities entail the robot to be endowed with different safety (hardware and control) settings while executing tasks.

In FourByThree  four different pilots are taken into account covering different production processes, i.e. assembly/disassembly of parts, welding operations, large parts management and machine tending. Among these, the ALFA Pilot is particularly relevant from the HRC perspective. This case study corresponds to a production industry (the ALFA Precision Casting222ALFA is a medium sized company producing aluminium parts for different industries for applications that are characterized by low size production batches and requiring tight tolerance and dimensional precision.) which represents a real working scenario with different relevant features (e.g. space sharing, collaboration or interaction needs). The overall production process (summarized in Fig. 6.7) consists of a metal die which is used to produce a wax pattern in a injection machine. Once injected, the pattern is taken out of the die. Several patterns are assembled to create a cluster. The wax assembly is covered with a refractory element, creating a shell (this process is called investing). The wax pattern material is removed by the thermal or chemical means. The mould is heated to a high temperature to eliminate any residual wax and to induce chemical and physical changes in the refractory cover. The metal is poured into the refractory mould. Once the mould has cooled down sufficiently, the refractory material is removed by impact, vibration, and high pressure water-blasting or chemical dissolution. The casting are then cut and separated from the runner system. Other post-casting operations (e.g. heat treatment, surface treatment or coating, hipping) can be carried out, according to customer demands.

Given this production process, the first step (preparation of the die for wax injection and extraction of the pattern from the die) has a big impact on the final cost of the product, and it represents a relevant application scenario. Thus, the involvement of a collaborative robot has been envisaged to help the operator in the assembly/disassembly operation. The operation consists of the following steps: (i) mount the die; (ii) inject the wax; (iii) open the die and remove the wax; (iv) repeat the cycle for a new pattern starting back from step (i). The most critical sub-operation is the opening of the die because it has a big impact on the quality of the pattern.

6.3.2 Assembly/Disassembly Operation

Due to the small size of the dies and the type of operations done by the worker to remove the metallic parts of the die, it is very complex for the robot and the worker to operate on the die simultaneously. Figure 6.8 shows some of the steps of the overall (manual) operation. However, they can cooperate in the sub-operations concerning the assembly/disassembly of the die. Once the injection process has finished, the die is taken to the workbench by the worker. The robot and the worker unscrew the bolts holding the top cover. There are nine bolts, the robot starts removing those closer to it, and the worker the rest. The robot unscrews the bolts on the cover by means of a pneumatic screwdriver. The worker removes the top cover and leaves it on the assembly area (a virtual zone that will be used for the re-assembly of the die). The worker turns the die to remove the bottom die cover. The robot unscrews the bolts on the bottom cover by means of a pneumatic screwdriver. Meanwhile the operator unscrews and removes the threaded pins from the two lateral sides to release the inserts. The worker starts removing the metallic inserts from the die and leaves them on the table. Meanwhile, the robot tightens the parts to be assembled/reassembled together screwing bolts. The worker re-builds the die. The worker and the robot screw the closing covers. The human and the robot must collaborate to perform assembly/disassembly on the same die by suitably handling different parts of the die and screwing/unscrewing bolts. Specifically, the human worker has the role of leader of the process while the robot has the role of subordinate with some autonomy.

6.4 Dynamic Task Planning in FourByThree

In FourByThree and more in general in HRC applications, the envisaged dynamic task planning system must realize a human aware planning and execution mechanism capable of allowing a robot to safely interact with an operatore. The control mechanism must adapt robot plan and motions according to the expected and observed behaviors of the related human operator [Cesta et al., 2016, Pellegrinelli et al., 2017]. In this sense, the dynamic task planning system applies and extends the hierarchical timeline-based modeling approach by introducing supervision and coordination issues. Supervision models the operational requirements of the production processes. It models the high-level tasks to perform in order to complete the process and the related precedence constraints that must be satisfied. Coordination models the possible decompositions of the high-level tasks in low-level tasks the human and the robot can directly perform and the possible assignments. Moreover, the human is an active element of the environment that cannot be directly controlled by the robot and therefore the human is modeled as a variable of the domain whose values (i.e. the low-level tasks the operator can directly perform) are all uncontrollable. The dynamic task planning framework must plan for the tasks the human and the robot must perform by coordinating them and by taking into account the temporal uncertainty of the human. Human activities are uncontrollable and therefore the system must generate and execute plans without making any hypothesis on the actual duration of the tasks assigned to the human.

6.4.1 Task Planning Model for Assembly/Disassembly

Figure 6.9 shows the hierarchical structure of the control model for the assembly/disassembly production process of the ALFA pilot of the project. The supervision layer represents the elements describing the processes of the work-cell. The ALFA state variable modes the general ALFA pilot and the related processes. Specifically, each value of the state variable represents a specific process (e.g. the Assembly operation) the human and the robot can perform in the pilot. The AssemblyProcess state variable models the assembly/disassembly operation by specifying the set of high-level tasks required. As shown in Figure 6.10 a set of constraints specifies the operational requirements that guarantee a correct execution of the process. For example, these requirements may specify ordering constraints between the high-level tasks or may specify different procedures for performing the process (e.g. alternative sequences of high-level tasks). In this specific case, the operational requirement of the supervision layer specifies a total ordering among the high-level tasks composing the Assembly process of the case study.

It is worth observing that the control model is not considering coordination features at this abstraction level. Each high-level task represents a complex procedure which must be further decomposed in (primitive/atomic) low-level tasks the human and the robot can directly handle. Some primitive tasks can be performed either by the human or by the robot and it is up to the task planner deciding who must execute them. Moreover, given a high-level task, the system must coordinate human and robot activities according to the type of collaboration desired for the specific collaboration scenario. Different types of collaboration entail different safety settings and therefore different configurations of the robot for performing tasks.

Figure 6.11 shows an example of coordination requirements between the robot and the human with respect to the high-level task named BaseRemoval of the Assembly process. The model describes the sequence of low-level tasks needed to properly complete the BaseRemoval task with their assignments. The robot and the human simultaneously unscrew the bolts of the base of the die and therefore the type of collaboration required is simultaneous in this specific case (the human and the robot work on the same workpiece while performing different tasks, i.e. unscrewing bolts). Again, the control system (and the robot) must be aware of the human and adapt its tasks according to the human-robot collaboration process defined.

Finally, the low-level tasks of the robot must be further decomposed in order to synthesize the set of commands/signals to be dispatched for execution. For example, the Screw task of the RobotController in Figure 6.11, requires to set the arm on the bolt to screw and then activate the tool (i.e. the screwdriver) in order to actually screw the bolt and complete the task. According to this description, the Screw task must be decomposed in terms of commands that allow the robot to assume the desired pose and activate/deactivate the tool. Specifically, the related synchronization rule of the model, constrains the behavior of the RobotArmController and the ScrewDriverController (see Figure 6.12) by specifying the values they must assume (i.e. tokens) and the related temporal constraints that must be satisfied. The OnTarget value of the RobotArmController sets the arm on the target bolt. The Operating value of the ScrewDriverController activates the tool in order to start screwing the bolt. The temporal constraints shown in Figure 6.12, allow the arm to keep the position for the entire duration of the task.

Figure 6.13 shows an excerpt of a hierarchical timeline-based plan for the Assembly process of the ALFA case study. The horizontal sections (i.e. bars with different colors) partition the plan according to the hierarchy depicted in Figure 6.9. The vertical section (in red) depicts an example of high-level task decomposition and application of the synchronization rules of the domain. Namely, the decomposition of the BaseRemoval high-level task of the Assembly process: the BaseRemoval task requires the human operator and the robot to simultaneously unscrew some bolts from two lateral sides of the work-piece, then the human should rotate the piece and finally, the operator and the robot unscrew bolts from two lateral sides of the piece. Figure 6.13 shows that the plan satisfies the production requirements of the high-level task. Indeed, a synchronization rule requires that the low-level tasks for unscrewing bolts should be executed during the BaseRemoval task. Moreover, the first unscrew tasks must be performed before the operator rotates the piece, while the second unscrew tasks must be performed after the operator rotates the piece. It is also possible to observe that robot’s tasks are further decomposed in order to synthesize a more detailed representation of the activities the robot must perform to actually carry out the low-level tasks. For instance, the robot must set the arm on a specific target and then must activate the tool in order to perform an unscrew operation. Again, in Figure 6.13, a during temporal constraint holds between the Unscrew low-level task token and the OnTarget and Operating tokens.

6.4.2 Feasibility Check of the Task Planning Model

Deliberation time i.e. the time spent by the dynamic task planning system to generate a plan for the considered production process, has been considered as the first key performance indicator to be assessed in order to test the performance of the dynamic task planning system. Thus, with respect to the planning model of the assembly/disassembly process described in the previous section, different planning scenarios have been considered by varying the complexity of the dimensions of the problems:

• •

Production process complexity - three different production procedures have been analyzed by taking into account an increasing number of tasks needed to complete the assembly/disassembly process: the small procedure consists of 6 tasks; the medium procedure consists of 10 tasks; the large procedure consists of 15 tasks.

• •

Human-Robot effort - for each production procedure an increasing involvement of the robot has been considered in order to increase the number of tasks the robot must perform to complete the process and consequently decrease the effort of the human

Figure 6.14 shows the deliberation times of the dynamic task planning system for the considered scenarios. In general, the higher is the number of tasks needed for the process, the higher is the number of tasks that can be assigned to the robot and, consequently, the higher is the complexity of the resulting problem with respect to deliberation. As the results in Figure 6.14 show, an increasing complexity of the scenario entails higher deliberation times. Nevertheless, planning costs result to be compatible with the latency of the production environment. Indeed, the performance is compatible with the latency usually involved in this type of manufacturing applications. In particular, the experimentation emphasizes the flexibility of the envisaged approach to planning which is capable to adapt the assignment and coordination strategies to different human-robot collaboration settings.

6.4.3 The Dynamic Task Planning Module in Action

The dynamic task planning system has been tested on a ROS-based simulator333*”The Robot Operating System (ROS) is a collection of tools, libraries, and conventions that aim at simplifying the task of creating complex and robust robot behavior across a wide variety of robotic platforms”* - from: http://www.ros.org/about-ros/ which provides an implementation of the functional control level of a generic robotic arm. Figure 6.15 shows the process architectural view [Kruchten, 1995] of the dynamic task planning module describing the main elements composing the module at runtime (i.e. the processes) and their interactions. The process architectural view aims at describing how the control flow is structured and how the deliberative and executive processes (both relying on EPSL-based planning and execution capabilities) interact. In particular, this view describes the management of plan execution failures and the related replanning mechanism. The execution decisions of the dynamic task planning module, shown in Figure 6.15, are taken on the fly during execution. Without an execution policy a valid plan may fail due to wrong dispatching or environmental conditions (controllability problem [Vidal and Fargier, 1999]) and therefore replanning is the basic mechanism which allows the module to deal with exogenous events and complete the execution of a plan.

The processes of the dynamic task planning module exchange information concerning the task to be managed through queues. The different queues of Figure 6.15 represent the states composing the task lifecycle within the module. The task lifecycle models the control flow of the dynamic task planning module. The buffered queue is the entry point of the module, it contains the high-level task requests the module must manage i.e. requests of performing a particular process of the factory (e.g. the assembly/disassembly process).

The Deliberative process takes a task request from the buffered queue and synthesizes a (pseudo-controllable) plan for the task. The generated plan represents a suitable set of (low-level) tasks the human and the robot must execute according to the desired operational requirements. Thus, the task (i.e. the high-level task request) is ready for execution and therefore the task with the related plan is added to the planned queue.

The Executive process takes a task request from the planned queue and starts executing the related plan by sending commands to the system (or a ROS-based simulator) through the Dispatcher and receiving feedbacks about command execution through the Monitor. As described in Section 6.2.1, the Dispatcher is responsible for deciding the start of token execution according to their controllability properties (see Section 6.2.3). The Monitor is responsible for managing execution feedbacks from the environment (or the ROS-based simulator) in order to verify the correctness of the plan with respect to the actual behavior of the system. If no inconsistency is detected the execution continues until the plan is ended and the task request, with the resulting plan, is added to the executed queue. Otherwise, if an inconsistency is found then the executive interrupts the execution and the task request with the interrupted plan is added to the failure queue.

Execution fails every time the uncontrollable dynamics of the environment do not comply with the plan and the "expected" uncertainty of the domain. As described in Section 6.2.4, temporal flexibility allows the executive to capture an envelope of possible (temporal) behaviors of the environment. Different temporal behaviors can be easily managed by the executive by temporally adapting the plan, if such behaviors comply with the model. However, if the observed dynamics of the environment do not comply with the expected uncertainty then the plan cannot capture these behaviors and replanning is needed. For example, human tasks are (fully) uncontrollable and therefore the executive cannot make any hypothesis on their actual duration. The model provides an estimation of the durations of such tasks in terms of minimum and maximum expected duration. The deliberative generates plans according to this estimation (pseudo-controllable plans), thus the executive can execute them if the observed behavior of the environment complies with the model (i.e. the actual duration of uncontrollable tasks comply with the expected durations). Thus, if the observed duration of a human task is higher than the expected maximum, then the plan cannot be adapted and a new plan is needed in order to address the real situation.

The Failure Manager process is responsible for managing the interruption of the execution in order to set a stable state before generating a new plan. The process takes the interrupted task with the related execution trace (i.e. the executed portion of the plan) from the failure queue and interacts with the environment (the ROS-based simulator in this case) in order to set the robot in a stable state. As broadly described in Section 6.2.4, the Failure Manager analyzes the timelines concerning the primitive variables of the domain in order to set the situation the Deliberative will replan from. In this specific case, if the execution is interrupted while the robotic arm is moving, the Failure Manager waits the execution feedback of the motion in order to let the Deliberative start replanning with the robotic arm set in a stable position. Another possible approach would allow the Failure Manager to interrupt the motion and send the commands needed to set the robotic arm in a known (initial) position. In general, the logic implemented by the Failure Manager cannot be generalized because it is strictly connected to the specific robotic platform considered and the related functional level (i.e. the set of sensing and action primitives available for interacting with the robotic platofrm).

When a stable state is reached i.e. both the robot and the human are in a known stable state, the Failure Manager leverages the execution trace of the interrupted plan and the current situation of the robot and the environment to build the problem specification for the new plan. The interrupted task with the related problem specification is added to the replanning queue and the Deliberative starts generating a new plan by fitting the related problem specification.

Experimental evaluation on a ROS-based simulator

Figure 6.16 shows a screenshot of a simulation for the assembly/disassembly process. The left-hand side of Figure 6.16 shows a portion of the plan of Figure 6.13 during execution. It shows the Gantt chart representing the timeline of the Human (the sequence of red tasks), the timeline of the RobotController (the sequence of blue tasks) and the timeline of the RobotArmController (the sequence of green tasks). The right-hand side of Figure 6.16 shows a simple 3D model of the workcell composed by a robotic arm and the workpiece. The colored blocks of the workpiece represent the bolts the robot and the human are supposed to unscrew within the assembly/disassembly process. Specifically, the blue blocks represent the bolts the Deliberative has assigned to the robot, and the red blocks represent the bolts the Deliberative has assigned to the human.

Simulations have shown the capability of the dynamic task planning system of coordinating the human and the robot in order to perform assembly operations. The coordination takes into account the expected duration bounds of the tasks (especially the task of the human) that are assigned by taking into account the makespan of the plan. Namely, the dynamic task planning system generates timeline-based plans that minimize the expected duration of the overall process and therefore optimize/maximize the throughput of the factory. Simulations have also shown the capability of the dynamic task planning system of adapting the execution of robot tasks to the observed behavior of the human operator. Leveraging temporal flexibility, the system easily captures the possible behavior of the human by dispatching robot tasks accordingly. However, if the observed behavior of the human does not comply with the expected one (i.e. with the model) the system cannot proceed with the execution and the plan is interrupted (execution failure). In this case, simulations have shown the capability of the system of generating a new plan through replanning. Specifically, the Failure Manager sets the robot in a stable state by waiting for execution feedbacks of not completed motion tasks or any uninterruptible operation. Then, the Deliberative generates a new plan by taking into account executed tasks and reassigning missing tasks to the human and to the robot. Once the new plan has been generated, the Executive resumes plan execution starting with the reassigned tasks.

Chapter 7 Knowledge-based Control Loop for Flexible Controllers

The ideal robot is an artificial entity (an agent) capable of setting its own goals and of planning actions to achieve them. The research community in robotics and AI has been building many types of robots applying different techniques but yet is still far from anything ideal. There is still a limited understanding of what are the essential characteristics of artificial agent like robots. From a local point of view, robot software or hardware parts and modules, like sensors, reasoning engines, etc., present several limitations when compared to the capabilities of similar parts in the human being. Similarly, from a global point of view, there is not a clear vision of how to integrate different parts together in order to realize an agent capable of autonomously operate in the environment by understanding the current situation and "act" accordingly by properly manage the dynamics of the "world". These philosophical problems are not just theoretical but they are also present in practical and specific areas like industrial robots. In particular there are several research initiatives that focus on the construction of robots that can quickly adapt changes in the production environments [Wiendahl et al., 2007]. Traditional systems, based on centralized or hierarchical control structures, like the plan-based approach described in the previous chapter, typically require major overhauls of their control code when some sort of system adaptation and reconfiguration is required.

Dynamic working environments like Reconfigurable Manufacturing Systems (RMSs) [Koren et al., 1999] require control processes with an high level of flexibility. The actual capabilities of an agent and even the production processes of the factory may change quickly in such contexts. Different configurations of the shop-floor or the introduction of different production goals may change the type and/or the ways agents carry out their tasks. Classical plan-based controllers usually rely on a well-defined and static model of the world which could become obsolete very soon. The domain model would require a great design effort to be as stable as possible and a continuous maintenance which would negatively affect the productivity of the factory. The pursued solution is to extend classical plan-based control architectures by introducing knowledge representation and reasoning mechanisms into the control loop. Semantic technologies provide the flexibility needed to dynamically adapt the control model (i.e. the planning model) to different production settings. Thus, this chapter presents the Knowledge-based Control Loop (KBCL) which proposes an extension to classical plan-based control architectures suitable for artificial agents in general and robotics in particular. The proposed solution relies on an ontological approach for knowledge classification and management structuring information about the capabilities of an agent and the related working environment (e.g. an industrial robot in a manufacturing environment). This chapter describes how the knowledge of the agent is structured and how such knowledge can be exploited to dynamically generate a timeline-based planning model used to plan and executive the activities of the agent.

7.1 Flexible Plan-based Control Architectures

The integration of knowledge reasoning and planning is today critical. Indeed, the integration of these two technologies involves the manipulation of symbolic information at different levels of abstraction and its translation into different structures for controlling the state of the different components of the agent and, consequently, their interaction with the environment.

Despite the variety of uses of ontologies in robotic applications, the organization and management of the information needed to act at run-time remains an open problem. This is a challenging problem to face in order to develop adaptive autonomous robots. The pursued solution aims at integrating knowledge reasoning and planing techniques in order to provide the control process with the flexibility needed to dynamically adapt the control model to the actual state of the system and the environment. The proposed approach relies on two elements: a foundational ontology which organizes the information, and control process which continuously updates data and manages the flow of information needed to plan and execute activities of the agent.

The ontology defines the structure of the general information the flexible control module must deal with. The ontology provides a semantics for the concepts and the general properties characterizing the application domain. The control process leverages the ontology to generate and manage the knowledge of the specific agent to control. Specifically, the ontology guides the interpretation of data concerning the agent and the environment, and allows the control process to dynamically instantiate such information into a Knowledge Base (KB) which describes the specific capability of the agent and the specific working environment. On the basis of the obtained KB, the control process can dynamically generate the planning model tailored to the actual state of the the actual state of the agent and the related working environment. Then, the control process continuously monitor the agent and the environment in order to maintain the KB and also the control model updated.

7.1.1 The Manufacturing Case Study

The flexible control architecture described in this chapter has been designed in order to work in the context of a pilot case from the GECKO project [Borgo et al., 2014a]. The pilot case consists in a Reconfigurable Manufacturing System (RMS) for recycling Printed Circuit Boards (PCB). The plant is composed of different machines for loading/unloading, testing, repairing and shredding of PCBs and of a conveyor system that connects them. The conveyor is implemented through a Reconfigurable Transportation System (RTS) composed of a set of reconfigurable mechatronic components, called Transportation Modules (TM), see Fig. 7.1. The goal of the plant is to analyze defective PCBs, to automatically diagnose their faults and, depending on the type of the malfunctions, attempt an automatic repair or send them to waste.

The proposed agent architecture is wrapped around each of the TMs hence its functionalities are introduced with more details. Each of the TMs combines three Transportation Units (TUs). The units may be unidirectional or bidirectional, with bidirectional units enabling also movements from side to side (cross-transfers) from/to other TMs, see Fig. 7.1. The TM can support two main transfer services, forward and backward, and zero to many cross-transfer services. Different configurations can be deployed varying the number of cross-transfers components and thus enabling multiple I/O ports. TMs can be connected back to back to form a set of different conveyor layouts.

The manufacturing process requires each PCB to be loaded on a fixturing system (a pallet) in order to be transported by the TMs and processed by the various machines of the RMS. The transportation system can move one or more pallets (i.e., a number of pallets can simultaneously traverse the system) and each pallet can be either empty or loaded with a PCB. At each point in time a pallet is associated with a given destination and the RTS allows for a number of possible routing solutions. The next destination of a pallet carrying a PCB can change over time as operations are executed (e.g., by the test station, the shredding station, the loading/unloading cell). The new destination is available only at execution time.

The GECKO proposal was to realize a distributed control infrastructure composed by a community of autonomous agents [Borgo et al., 2014a] able to cooperate in order to define the paths the pallets must follow to reach their destinations. Thus, these paths are to be computed at runtime, according to the actual status and the overall conditions of the shop floor, i.e.. no static routes are used to move pallets. The decisions of the coordination algorithm [Carpanzano et al., 2016] act as goal injection for the planning mechanism of each agent. Hence according to our pursued abstraction, the plant is a set of TMs endowed with independent capabilities to carry on their goals, by analyzing the current situation, synthesizing a planning domain and problem, then planning and executing the plan for such goals.

It is worth observing that a plan-based controller can endow an agent with the desired autonomy (i.e., deliberative capabilities), but given the particular dynamic nature of RTSs it does not guarantee a continuous control process capable to face all the particular situations/configurations. Indeed it is not easy (or hardly possible) to capture all the dynamics of the production environment in a unique planning domain. The specific capabilities of a TM in the RTS are affected by many factors, e.g., a partial failure of the internal elements of a TM, a reconfiguration of the RTS plant or maintenance activities of some TMs of the plant. Thus, it is not always possible to design a plan-based controller which is able to efficiently handle all these situations. The higher is the complexity of the planning domain the higher is the time needed to synthesize the plans and the latency of the control architecture must be compatible with the latency of the plant.

Thus, the key direction in GECKO project has been the one of endowing the plan-based controller of a TM (i.e. an agent) with a knowledge reasoning mechanism capable to build the actual state of the production context by dynamically inferring the actual capabilities of the TM with respect to the detected configuration of the plant. In this way, the plan-based controller can automatically generate and continuously maintain updated the timeline-based model of the TM according to the inferred knowledge.

7.1.2 The Use of Ontologies in Manufacturing

In robotics and more generally in manufacturing, the use of ontologies is crucial to improve the adaptability and the flexibility of classical approaches [Turaga et al., 2008]. In several works, ontologies have been exploited to design more autonomous, flexible, adaptive and proactive artificial agents. Since researchers have applied ontologies to solve or at least mitigate a variety of problems, applications differ in their assumptions and goals.

In [Suh et al., 2007], a Robot knowledge framework (OMRKF) is exploited, OMRKF contains a series of ontology layers, includinga robot-centered and a human-centered ontology. Beside a perception layer, needed for the sensory data, the system is composed by an object layer (model), a context layer and an activity layer. The framework lacks of a foundational approach as can be seen in the object classification where, for example, the "living room" is classified as a space region and not as the role of the region (the problem becomes clear by observing that a region of space is fixed while the living room can be located in different parts of the building at different times, and can even disappear from the building).

Relatively to the connection between the KB and the planning module, the work [Hartanto and Hertzberg, 2008] exploits a model filtering approach based on a Hierarchical Task Network (HTN). The agent’s knowledge of the environment is stored in a fixed ontology and some filters on this knowledge are set up. Given a planning task, the system selects one of the filters to isolate a suitable subset of the system’s knowledge and uses this subset to constrain the plan by deleting non-reachable constants. While this technique can be efficient in terms of plan adaptation, the knowledge is only filtered, thus cannot be augmented nor modified, not even contextualized to the specific problem.

Other research projects, like KnowRob [Tenorth and Beetz, 2009] and ORO [Lemaignan et al., 2010] focus on learning and symbol grounding and use ontologies for obtaining an action-based knowledge representation able to support cognitive functionalities. At the ontological level, these knowledge systems show problems similar to those discussed earlier (e.g. functionality is confused with activity so that it is not possible to "discover" new ways to perform a function).

7.2 Knowledge and Plan-based Control in a Loop

The Knowledge-based Control Loop (KBCL) represents the envisaged flexible control architecture which integrates a knowledge processing mechanism with planning and execution in order to dynamically generated and adapt the timeline-based model needed to actually control an agent (i.e. a TM of the plant in the GECKO project). Figure 7.2 shows the key integration of distinct cognitive functions composing the architecture. At a higher abstraction, the figure shows the integration of two "big boxes" called here Knowledge Manager, that contains the know-how of the agent, and Deliberative Controller that represents the EPSL-based controller which plans and executes the activities of the agent. To make the whole idea operational we need to open the boxes and describe what is needed to allow the two functionalities to work together.

The goal of KBCL is to have a coherent and continuous flow of information from the Knowledge Manager to the Deliberarive Controller and to extend the capabilities of the overall system by exploiting reasoning capabilities. Following a careful analysis of the reasoning needs, the Knowledge Manager relies on a suited ontology which models the general knowledge of manufacturing environments. The ontology contains (i) a classification of relevant information in three distinct Contexts – namely Global, Local and Internal (see later) – and (ii) a Taxonomy of Functions which classifies the set of functions the agents can perform according to their effects in the environment (see later).

The Knowledge Manager exploits the ontology to build and manage the KB of the particular agent to control. The KB represents an abstract description of the structure and the capabilities of the agent and also of the production environment (from the agent’s point of view). Namely, the KB represents the "instantiation" of the general knowledge to the particular agent to control. In this context the Rule-based Inference Engine is a specific module which is responsible for processing KB information by inferring additional knowledge about the functional capabilities the agent is actually able to perform (see later for further details). Thus, given a TM of the RTS of the case study, the KB contains information concerning the devices that compose the TM (e.g. the cross transfers, the conveyor engines, the port sensors), the set of other TMs and/or working machines directly connected (i.e. the set of collaborators) and information concerning the whole production environment from the agent perspective (e.g. the topology of the shop floor). Then the Inference Engine analyzes the structure of the TM and its collaborators in order to add to the KB information about the set of transportation functions the TM is actually able to perform.

The Planning Framework provides timeline-based deliberative features relying on the planning model generated from the KB to actually control the mechatronic device. More specifically, it is a wrapper of the planning and execution system employed to provide deliberative capabilities. It is responsible to integrate KB information with planning by automatically generating the model of the mechatronic device to control. Indeed, planning domain and problem specifications are dynamically generated from the KB and an off-the-shelf planning and execution system is activated to synthesize signals for the actuators that control the mechatronic device. The Mechatronic Module is the composition of a Control Software and a Mechatronic Component (e.g., a transportation module, a working machine, etc.). In our case the control software is based on standard reference models (e.g., IEC61499) and each mechatronic component is then represented by dedicated hardware/software resources encapsulating the module control logic.

The Knowledge-based Control Loop (KBCL) represents the overall process which allows the integration of the elements described above in a unique control infrastructure. The resulting control process enables an agent to dynamically represent its capabilities, the detected environmental situation and to infer the set of available functions on which a coherent planning model is generated.

7.2.1 The Knowledge-based Control Loop at Runtime

The management of the KB, the generation of the planning domain, the continuous monitoring of the information concerning the agent and the environment, represent the rather complex activities the KBCL process must properly manage at runtime. In this regard, the KBCL process consists of the following phases: (i) the setup phase; (ii) the model generation phase; (iii) the plan and execution phase; (iv) the reconfiguration phase.

The setup phase generates the KB of the agent by processing the raw data received from the Mechatronic device through a Diagnosis Module. The resulting KB completely describes the structure of the particular module to control, the set of TMs the module can cooperate with and the set of functions the module is actually able to perform in order to support the production flow. Then, the model generation phase exploits the KB of the agent to generate the timeline-based planning model the Deliberative Controller needs to actually control the device.

When the planning domain is ready the planning and execution phase starts, and the Deliberative Controller continuously builds and executes plans. During this phase the KBCL process behaves like classical plan-based control systems. The Planning Framework builds the plan according to some tasks to perform. Soft changes in the plan execution are directly managed by the Deliberative Controller, e.g.  temporal delays of some planned activities. Conversely whenever the Diagnosis Module detects a structural change of the agent and/or of its collaborators e.g. a total or partial failure of a cross transfer of the TM to control (i.e. Hard changes), the reconfiguration phase starts.

The reconfiguration phase determines a new iteration of the KBCL process cycle. The KB of the agent is updated by detecting the new state of the mechatronic device and its production environment as well as inferring the updated set of functions the TM can perform according to the new state. As before, once the KB of the agent is complete, the planning model of the Deliberative Controller is also updated with respect to the new state of the module. It is worth observing that the KB and the planning model are updated only when structural changes that impede the execution of the plan are detected.

7.3 Modeling Knowledge with Ontology and Contexts

In a changing environment the agents must coherently share information relevant to the tasks. Thus an ontological analysis allows to build reliable systems that exploit different information types and contexts. The aimed generality lead to a structure a that is neither tailored to a specific type of agent nor to a specific type of situation. It is not based on an information model at the enterprise or shop floor level nor developed for some specific type of action. The result is a general mechanism to dynamically generate a high-level description of agent’s capabilities and system’s situations.

The first result of this analysis is the separation of two layers of information: organizational knowledge and factual knowledge. The organization knowledge is the foundational knowledge, i.e., the knowledge about the basic assumptions in the domain like the notion of object, agent, production, etc., including their relationships. This knowledge fixes what kind of entities, events and interactions there can be in general. Factual knowledge, instead, identifies how the actual scenarios is, out of all the possible configurations: which objects are presents and where, which actions are executed and by which agent, which changes occur and to which object. Factual knowledge can be extended (without changing the foundational knowledge) as needed, e.g., to include knowledge about new devices (tools, machines) or changes in the shop floor layout. Changes in these two parts of the knowledge framework follow different principles and have different consequences. By keeping them apart, we can make them interoperate covering all the knowledge needed in the production systems [Chandrasegaran et al., 2013].

For the organizational knowledge the proposed approach relies on the foundational ontology DOLCE  the Descriptive Ontology for Linguistic and Cognitive Engineering [Masolo et al., 2002]. This is a domain-independent top-level ontology that has been exploited at different levels in the engineering and industrial domains, e.g., [Borgo, 2014, Prestes et al., 2013, Borgo and Leitão, 2004]. DOLCE furnishes the basic structure of the knowledge the KBCL relies on and it will be enriched with domain knowledge, for instance adding the notions of artificial agent and engineering function. The knowledge framework available to an agent, will be an extension of this ontological system. Since DOLCE is based on a first-order language with formal semantics, the ontology and the resulting knowledge base can be exploited via automatic reasoning

7.3.1 The DOLCE Ontology

The DOLCE ontology is a formal system built according to an explicit set of philosophical principles that guide its use and extension [Masolo et al., 2002]. DOLCE focuses on particulars, as opposed to universals. Roughly speaking, a universal is an entity that is instantiated or concreted by other entities (like the property "being a tool" or "being a production process"). A particular, an element of the category particular, is an entity that is not instantiated by other entities (like Eiffel Tower in Paris or Barack Obama). particular includes physical entities, abstract entities, events and even qualities as shown below.

The DOLCE ontology formalizes the distinction between things like a car and an organization (this category is called endurant), and events like transporting by means of a car and resting (category perdurant), see 7.3. The term "object" is used in the ontology to capture a notion of unity as suggested by the partition of the category physical endurant (a subcategory of endurant) into categories amount of matter, like the plastic with which a water bottle is made, physical object, like a car, and feature. Features are entities that existentially depend on other objects, e.g., a bump on a road or the workspace for a robotic arm. There are other two subcategories of physical object, namely, agentive physical object, e.g. a person, and non-agentive physical object, e.g., a drill.

DOLCE also provides a structure for individual qualities (elements of the category quality like the weight of a given car), quality types (weight, color and the like), quality spaces (spaces to classify weights, colors, etc.), and quality positions or qualia (informally, locations in quality spaces). These, together with measure spaces (where the quality positions get associated to a measure system and to numbers), are important to describe and compare devices and processes. The exact list of qualities may depend on the entity: shape and weight are usually taken as qualities of physical endurants, duration and direction as qualities of perdurants. An individual quality, e.g., the weight of an hammer, is associated with one and only one entity; it can be understood as the particular way in which the hammer instantiates the general property "having weight". That individual weight quality is what can be measured when the hammer is put on a scale (if we put another hammer, no matter how similar, another individual quality would be measured, i.e., that of the second hammer even if the scale indicates exactly the same value). The change of an endurant in time is explained in DOLCE through the change of some of its individual qualities. For example, with the substitution or damaging of a component, the value of the weight quality of a car may change.

DOLCE’s taxonomic structure is depicted in Figure 7.3. Each node in the graph is a category of the ontology. A category is a subcategory of another if the latter occurs higher in the graph and there is an edge between the two. particular is the top category. The direct subcategories of a given category form a partition. In the graph, dots indicate that not all the subcategories of that category are listed. Some relations are particularly relevant in this context, e.g., the parthood relation: "x is part of y" (written: P(x,y)), with its cognates the proper part (written: PP(x, y)) and overlap relations (written: O(x, y)). It applies to pairs of endurants (e.g., the joint is part of the robotic arm) as well as to paris of perdurants (e.g., riveting is part of the assembling process). On endurants parthood has an additional temporal argument since and endurant may loose or gain parts throughout its existence (e.g., after substituting a switch in a radio, the old switch is not part of the radio). Another important relation is constitution, indicated by K: K(x, y, t) stands for "entity x constitutes y at time t", e.g., the amount of iron x constitutes the robot y at time t (this relation allows to say that part or all the iron x may be substituted over time without changing the identity of robot y like when substituting a worm component).

7.3.2 Ontological interpretation of Agents and their Environment

Recently there has been an increasing interest in the ontological modeling of artificial agents, and robots in particular [Prestes et al., 2013], which led to an IEEE standard (ORA – Ontologies for Robotics and Automation). Today’s approaches to robot modeling are interesting but further work si needed. For instance, it is unsatisfactory to take the characterization "being a robot" as a role (this is the choice in the IEEE standard ORA) since this implies that robots are such only when active, i.e., they appear and disappear by switching them on and off. While this avoids the problem of characterizing the essence of robots, the choice goes against common-sense. Robots do not seem to qualify as agentive entities in the strong sense since they lack intentional states, and it is dubious if they even qualify in the weak sense in most cases they have only conventional stimulus-response behavior. Up to today, any attempt to draw the line between artefactual tools and robots has met important criticisms. The following sections propose an extension of the DOLCE ontology to include robots, robotic parts and tools. The goal of this extension is to start from the notions of artefact and of agent, as introduced in foundational ontologies, and to propose a way to descriminate among types of artefacts as needed to model industrial scenarios.

Ontological speaking, following the analysis in [Borgo and Vieu, 2009], a robot is an artefact: it is intentionally selected (via construction) and has attributed technical capacities. Technical capacities can vary considerably depending on the robots: they can be quite limited, like in ant robots, or flexible and multipurpose like in industrial or humanoids robots. Since the focus is on industrial settings, thus on robotic arms, transportation modules and the like, the modeled robots are actually technological artefacts [Borgo et al., 2014b]: they are manufactured by following precise production plans and selected via dedicated quality tests. Thus, from the formal viewpoint industrial robots can be classified as (technological) artefacts i.e., elements of the artefact subcategory of non-agentive physical object [Borgo and Vieu, 2009].

The typical robots in the production scenarios are rational, reactive and may present some degree of autonomy. Today, they are rarely adaptive and embedded although these are desirable features. They can also be proactive: they have goals, typically provided by the production system to which they belong, and can sometimes choose, or at least reschedule, plans to optimize their achievements. In short, these robots are artefacts whose behaviors resemble agents’ behavior for the same goal(s). Since this behavior is expected from them, we propose to see a robot as an artefact whose attributed quality is to behave agent-like. It is important to point out that this modeling choice keeps agents and robots apart: a member of the latter group just mimics agents. The behavior can range from basic stimulus-response actions to activities controlled by sophisticated planning and goal adaptations, depending on what kind of agentivity the robot can behaviorally simulate. This is definitely acceptable for today’s robots and it does not exclude that future generations of robots might be considered as full-fledge agents.

The rest of the section refers to robots as agents. The symbol robot is used for the predicate "being a robot" and BehSp for the generic space of behaviors. Specifically, using the language of DOLCE from [Masolo et al., 2002, Borgo and Masolo, 2009, Borgo and Vieu, 2009], it is possible to formally model the ontological status of robots as follows:

[TABLE]

[TABLE]

The first formula says that a robot is an artefact. The second states what distinguishes a robot from other artefacts: the capacity attributed to the robot (AttribCap(a) $\land$ qt(a,r)) has values (ql(v, a, t)) that belong to the space of behaviors (Loc(v, BehSp))111The existence of quality a is enforced by formula 7.1 and the theory [Borgo and Vieu, 2009]. The characterization of the space of behaviors is stil under investigation.

Robot’s parts are themselves artefacts, thus elements of the artefact category. These are typically not robots, so their attributed qualities are of different types. The main distinction here is between the parts that are components, i.e. that constitute the robot like the engines that move the robotic arm structure and the structural pieces that are moved by the engines; and the parts that are tools used by the robot like the different types of gripper that can be substituted depending on the task to execute. These types of parts are isolated for their functional or structural contribution. There are, of course, also arbitrary parts like the upper half of the skeletal frame, which do not have special properties or functionalities and thus are not relevant in terms of knowledge and planning.

Components (tools) can be in an active/inactive (available/non-available) state for the robot. Sensors are listed among the components but the proposed characterization does not distinguish between sensors and actuators since these are seen as roles of the agent’s components (a drill can play both of them at the same or at different times). Finally, an object that is a component is such until substituted (or dismantled) while a tool may remain such even if substituted.

In the case of agents, the environment represents the area of interest in which the agent could act. For artificial agents, the environment might also include the requirements and specifications about the software components and their development. Since the reasoning mechanism deals with languages and software constraints in terms of contexts, the considered notion of environment focuses on the notion of location. Thus, at each point in time, the robot’s environment is described in terms of robot’s location including the elements the location contains plus entities that, even though not in the location, can interact (positively or negatively) with the robot’s activities and goals.

This view is fairly general and assumes that the environment depends on the robot’s features as well as on the features of the other entities. It is important to point out that the environment can change whenever the robot or its location or the entities there change. In the case of production scenarios, the robot’s environment can be identified with the collection of physical entities that are within a certain range from the robot (where the range may be bounded by physical barriers like floor, walls, ceiling, fences, etc). The environment is not necessarily limited to a precise region of space; it includes also entities with which the robot can interact in some ways (e.g., via wireless communication). In ontological terms, the environment is a compound physical object composed by all the physical objects that are within the interaction range (workspace) of the robot. The location of the environment corresponds to the location of the objects in the environment plus the locations reachable by the robot itself 222The location is fixed for robots like robotic arms, it is parametric (in particular, it may depend on the task) for mobile robots.

7.3.3 Ontology and Engineering of Functions

The classifications of the robots, the physical entities that may interact with them and their environments take care of the "static" part of the problem. Since a robot is supposed to act in order to reach its goals, it must also have the conceptual machinery to know what it can do and how, thus to plan its actions. In this regard, reasoning on (engineering) functions is unavoidable. The formalization of functions in robotics is rarely addressed and is too often confused with the notion of action, i.e., the performance of a function.

To overcome this problem, the proposed approach extends the DOLCE ontology with an ontology of high-level functions. This function ontology is integrated, via DOLCE, with the ontology of the robot and robot’s parts making it possible to model what a robot can do and how. Specifically, the interpretation of functions relies on the notion of function-as-effect (see Figure 7.4) which has been adapted borrowing from well-known functional approaches in engineering design like the FOCUS/TX [Kitamura et al., 2011] (for the distinction "what to" vs. "how to" and the notion of behavior), the Functional Basis [Pahl et al., 2007, Hirtz et al., 2002] (for the idea of a function list), and the Function Representation [Chandrasekaran and Josephson, 2000] (for the distinction between environment-centric and device-centric function). The guiding idea is to make it possible the identification of the high-level function (or sequence of functions) that need to be executed to reach a given goal. For this, it can be taken into account the difference between the actual state and the desired state, and identify the changes to be made. From this information, the robot can travel the taxonomy to identify the effects of the high-level functions and find a suitable combination.

Figure 7.4 show the top-level ontological functions organized in five brenches: functions to collect information, functions to change the operand’s qualities, functions to change the qualityrelationships, and functions to share information. For instance, "reclassify" stands for the function to change the classification of an operand, e.g. when, after a test, a workpiece is classified as malfunctioning; "change-over" applies when, e.g., a robot acts on itself to activate/deactivate some component; "channel" stands for the moving of an operand (change of its location); "stabilize" for maintaining relational parameters like when tuning electronic components to regulate the input-output relationship; "sense" for the operand testing function, i.e., to acquire information without altering the status or the qualities of the operand; finally, "send" stands for the function to output information like a signal that a workpiece is going to be transferred or that a failure occurred.

Of course, this information is not enough since it would model just the ideal capacities of the robot. Aiming to have a robot adapting its plan at run-time, we have to model the actual capacities of the robot, which implies to take malfunctioning and/or missing parts or even deteriorated behaviors into account. This information depends on the capacities of self-inspection built-in in the robot as well as on the possibility to compare the ideal action’s descriptions and the actual performances.

7.3.4 Context-based Characterization

An ontology is a conceptual tool used to structure information. Ontologies deal mainly with necessary information like the properties that an object must manifest (shape, weight, mass, etc) or the types of event (states, actions, processes and son on). Factual information, being information that depends on contingent data (like spatio-temporal location, agent’s setting, goals, etc), is generally characterized at the level of knowledge-bases. While this distinction might not be fully justified (and not even sharp), it remains important not to structure the ontology relying on factual knowledge. This principle is rarely recognized in applications and in particular in the development of ontologies for industrial application.

The distinction between necessary and contingent information concerns only the development of the ontology structure: it is important that factual information finds its place in the factory information system. This allows the system to classify and reason on factual information, for example, to understand the actual scenario and possible evolutions, to evaluate optimal production plans out of those that are actually possible, and even to establish the status of the resources or maintenance schedule. To act in real and evolving scenarios, factual information is thus essential. In the proposed approach, factual information is included in the KB (built on top of the ontology) and is organized into main categories called contexts. Contextualization enables to manage factual information with an ontologically sound approach. It gives also an advantage at the reasoning level: it allows to differentiate types of information depending on their usefulness in reasoning on a situation or task. After an ontological analysis based on [Borgo, 2007, Borgo and Masolo, 2009], it is possible to identify three contextual models dedicated to factual knowledge, and use them with the ontological framework. In particular, these contexts provide the time-dependent information needed to select how to execute high-level functions in the actual scenario.

The three context types are called global, local and internal, respectively. The global context collects information the agent cannot control nor modify like the shared language of the system, the agents present in the system, the system’s performance parameters. The local context collects information on the relationship between the agent and its neighbor elements (typically the human and artificial agents directly interacting with it), thus providing a local view of the topological setting. Finally, the internal context collects the information the agent has about itself as well as its capabilities towards itself (change-over) and towards the environment (communication and manipulation) [Borgo et al., 2015].

7.3.5 Applying Ontology and Contexts to the Case Study

Given a particular application like the manufacturing scenario of the case study, it is necessary to define the general knowledge the KBCL process must deal with in order to dynamically infer the specific capabilities of an agent and adapt the control model accordingly. Thus, the DOLCE ontology has been extended with the type of information needed by applying the context-based and the functional characterization described above.

Broadly speaking, the extended ontology aims at characterizing the knowledge concerning the general structure of a TM of the plant, the related working environment and the general functional capabilities of TMs in such a context. This information represents the general knowledge (i.e. the TBox) a KBCL process instantiates according to the specific features of the TM to be controlled, in order to generate the envisaged KB of the TM (i.e. the ABox).

Figure 7.5 shows the extension of the DOLCE taxonomy of particulars with respect to the non-agentive physical object category. According to the DOLCE interpretation of artifacts, robots and robot’s parts are modeled as subcategorires of artifact, as the taxonomy in Figure 7.3 shows. Robot’s parts can be further distinguished into robot’s components, i.e. parts that constitute the structure of the robot, and tools. These entities are modeled as subcategories of part. They represent artifacts with different attributed qualities with respect to robots as artifacts. Following DOLCE interpretation, the taxonomy can be extended by introducing the port, conveyor and cross transfer concepts as subcategories of component category, the concept of transportation module as subcategory of robot category.

The port, conveyor and cross transfer categories classify the elements characterizing the internal structure of a TM. The component category collects the elements that compose a robot. These components have a spatial location within the robot structure (this would not be enforced for tools since they can be external to the robot). Collaborating components for the Channel function must be spatially connected. In the case of the TM, the internal structure for this kind of functionality is determined by the connections of the components’ locations. The choice of modeling the elements of a TM with different categories rather then using the general component category, relies on the different properties these elements bring to implement functional capabilities (as it will be described in the next sections). The port category models the structural elements that allow a TM to connect with other TMs in its local contexts. These elements have a communication capacity which allows a TM to send and receive pallets to and from other TMs of the plant. The conveyor category models the engine elements that allow a TM to move pallets. They have a channel capacity which allows TMs to actually move a pallet between two spatial locations connected via that component. The cross transfer category models engine elements that allow a TM to change its physical configuration. They have a change over capacity which allows a TM to change its internal connections and enable the different paths the pallets can follow (internally).

The transportation module category characterizes TMs from a functional point of view. Namely, TMs are modeled as elements fo the robot category that can perform some channel functions and that have as components some elements of the port category, some elements of the conveyor category and some elements of the cross transfer category. In the manufacturing environment considered, elements of the channel category are functions that classify changes in the spatial location quality of an operand (i.e., a pallet). The execution of such a function changes the location of the pallet from the start location to the end location. The Figure 7.6 shows a graphical representation of the general class axiom defining the transportation module category.

The (working) environment of a TM is described in terms of the available collaborators. A TM collaborates with other TMs and machines of the plant by exchanging pallets through their connected ports. Thus, the subset of the plant’s agents that are directly connected to the TM and with which the TM can actually exchange pallets, constitutes the environment of the TM. In such a context, a collaborator is a relative concept which depends on the particular configuration of the TM considered. It represents a relationship between a TM and the directly connected agents. Thus, the concept of collaborator is modeled as a role that an agent, e.g., another TM, plays according to its local connections.

7.4 The Knowledge-Base Life Cycle

The Knowledge Manager module (KM) in Figure 7.2 is responsible for managing the lifecycle of the KB within the KBCL process. In the specific manufacturing case study, the KB models the particular TM to be controlled by specifying its internal structure, its connections with other TMs and the related functional capabilities. The management of the KB relies on a knowledge processing mechanism implemented by means of a rule-based inference engine which leverages a set of inference rules to build the KB of the agent.

The knowledge processing mechanism dynamically builds the KB elaborating raw data received from the Diagnosis Module and infers knowledge concerning the structure, the working environment and the functional capabilities of the agent. As Figure 7.7 shows, this mechanism involves two reasoning steps: the (i) the low-level reasoning step, and the (ii) high-level reasoning step. Specifically, these two steps iteratively refine the KB by combining a set of dedicated inference rules with the general knowledge of contexts and functions of the described ontology.

The first reasoning step, called the low-level reasoning, aims at characterizing the TM in terms of the components that actually compose the module (e.g., the ports, conveyors, etc.) and its collaborators. It leverages the internal and local contexts of the ontology as well as the classification rules to generate the initial instance of the KB which describes the structure of the agent and the related working environment. Thus, this initial KB describes the agent in terms of its internal and local contexts.

The second reasoning step, called the high-level reasoning, starts from the KB elicited after the previous step and generates further knowledge concerning the functional capabilities of the agent. Specifically, the high-level reasoning step relies on the taxonomy of functions and a set of domain-dependent inference rules, called capability inference rules, to complete the knowledge processing mechanism. The KB the high-level reasoning starts from, encodes the particular internal and local context of the agent. The inference mechanism can infer the set of functions the agent can actually perform by analyzing its structure and its working environment.

The output of the second reasoning step (and the overall knowledge processing mechanism), is the final KB which encodes a complete description of the structure of the agent, an interpretation of the working environment from the agent perspective and a description of the related functional capabilities of the agent. Such knowledge is then exploited in the KBCL process to generate the plan-based control model. The next two subsections provide a more detailed discussion of the two reasoning steps constituting the knowledge processing mechanism.

7.4.1 The Low-Level Reasoning Step

The low-level reasoning step is responsible for inferring information concerning the internal and local contexts of the TM. Namely, the result of this inference step is an initial KB describing the operating devices that compose the TM (i.e., the components) and the available collaborators. It builds an initial version of the KB by classifying data received from the Diagnosis Module on the basis of contexts categorization.

Input data consists of a set of individuals representing information about the parts that compose the TM, their connections and their capabilities. Figure 7.8 provides a (partial) graphical representation of a possible set of individuals (the nodes of the graph) and predicates (the edges of the graph) the knowledge processing mechanism receives from the Diagnosis Module. In particular, the figure shows the different contexts the individuals belong to, the reasoning step leverages to provide these data with additional semantics.

Given this set of data, the first rule the low-level reasoning step applies, aims at identifying the set of operative parts the TM can actually use to perform functions. These set of operative parts are represented as TM’s components. According to the ontological interpretation of the component category, a component is a structural part of a robot which has an operative state and may have some functional capabilities. The rule follows this functional interpretation of components and therefore, can be formally defined as follows:

[TABLE]

where $P(p,r)$ is a predicate asserting that the part $p$ is a structural element of robot $r$ and $hasCapacity(p,f)$ is a predicate asserting that the part $p$ has the functional capacity of performing some function $f$. According to the ontology, being $p$ a structural part of a robot $r$, with the capability of performing some function $f$, it is possible to infer that $p$ is an element of the component category. Consequently, the predicate component(p) is true.

The applied ontological approach models the different types of components that may compose a TM as Figure 7.5 shows. These components are modeled according to the different types of functional capabilities they have. Leveraging this interpretation, it is possible to define two additional rules that infer the specific type of component a particular part represents by taking into account the type of the associated functional capability:

[TABLE]

[TABLE]

The rules 7.4 and 7.5 infer conveyor and port components as structural parts of a robot, that have channel and communication capabilities respectively.

Given the components of the TM, the low-level reasoning step is completed by inferring the set of collaborators available. Similarly to components, the collaborators of TM are directly connected TMs of the plant that are in an operative state and therefore, can actually exchange pallets with the TM. The rule that allow to infer this information can be formally defined as follows:

[TABLE]

where $connection(l_{p},l_{c})$ is a predicate asserting that the location of the TM’s port $p$ is connected with the robot $c$. Figure 7.9 provides a (simplified) graphical representation of a possible KB resulting from the application of rule 7.6 (the dotted arrows represent the inferred properties concerning collaborators).

7.4.2 The High-Level Reasoning Step

The high-level reasoning step extends the KB elicited at the previous step to infer the actual capabilities of the TM on the basis of its current status and the current production environment. Given the information concerning components and collaborators of a TM, the first rule the high-level reasoning step applies, aims at inferring the primitive channels the TM can perform according to its internal structure. Indeed, operative components can be used by a robot to perform functions. With regard to TMs, a (operative) conveyor allows a TM to perform channel functions. According to this interpretation it is possible to define a rule to infer the set of primitive channels a TM can perform as follows:

[TABLE]

where conveyor$(c_{1})\land\ hasOpStat(c_{1},active)$ asserts that $c_{1}$ is a conveyor component of the TM whose operative state is active. Namely, the conveyor $c_{1}$ is an operative component of the Tm and therefore, it can be actually used to perform functions. Figure 7.10 shows a (simplified) graphical representation of the KB resulting from the application of rule 7.7. In particular, the figure represents predicates (the dotted arrows) and the individual (the "channel-1" onde) inferred and added to the KB.

The rationale of rule 7.7 relies on the functional interpretation of the conveyor category as the set of components that can perform channel functions. Thus, if an operative conveyor component connects two components of the TM through its spatial location, see the clause $connection(l_{2},l_{1})\land\ connection(l_{1},l_{3})$ in 7.7, then the conveyor can perform a primitive channel function between the components’ locations. Moreover, the $cConnect(l_{2},l_{3})$ is a transitive predicate which allows to "connect" and compose different channel functions. Indeed, if two spatial locations are connected through the $cConnect$ predicate, it means that there exists a composition of primitive channel functions that connect them.

A primitive channel involves components of a TM only. However, the channel capabilities the knowledge processing mechanism aims at inferring are those that involve the collaborators of a TM. Namely, channel functions that allow a TM to exchange pallets with other TMs of the plant. Such channels are called complex channels and can be inferred by applying the following rule:

[TABLE]

A key point of the rule 7.8 is that a complex channel function is interpreted as the composition of primitive channels the TM can perform internally. This is a quite flexible and general interpretation of a channel function. If one or more parts of a TM stop working (i.e., their operational status changes from active to inactive), then the TM will not be able to perform the related primitive channels and therefore, the high-level reasoning step will not be able to infer all the complex channels that depends on these parts. Similarly, if new components are added to the TM then, the high-level reasoning step will be able to inter additional primitive and also complex channels according to the resulting structure of the TM.

7.5 Generating the (Timeline-based) Control Model

A key role for the dialogue between the Knowledge Manager and the Deliberative Controller is played by the Model Generation process (Step 2 in Figure 7.2). The KB resulting from the knowledge processing mechanism provides an abstract representation of the capabilities, the structure and the working environment of the agent. The model generation process analyzes the KB to generate the related control model as a timeline-based planning specification of the agent.

The model generation process encodes the hierarchical modeling methodology described in Chapter 5 and builds the control model by leveraging the context-based characterization of the KB. The information concerning the global context and the taxonomy of function define the functional state variables that provide a functional view of the agent as a whole. These state variable describe the high-level tasks the agent can perform over time. The internal context contains structural information about the agent and therefore it is suited to generate the primitive state variables. These variables describe the behaviors of the physical/logical features that compose the agent. Usually the values of this type of variables directly correspond to states or actions the related domain features may assume or perform over time. The local context manages information concerning the working environment of the agent and therefore it is suited to build the set of external variables of the model. These variables model the collaborating agents (i.e. the directly connected TMs of the plant) whose behavior may affect the capabilities of the agent, even if not directly controllable.

Algorithm 8 describes the overall procedure of the generation process. Broadly speaking, the procedure consists of four specific procedures that analyze different areas of the knowledge about the agent in order to generate different parts of the control model. The procedure starts by extracting information related to the agent and initializing the P&S model (rows 3-4). According to the hierarchical timeline-based approach described in Chapter 5, a set of functional, primitive and external state variables is generated (rows 6-8). Finally, the hierarchical decomposition of functional values (i.e. values of functional state variables) is described by means of a suitable set of generated synchronization rules (row 10). The resulting timeline-based model is then composed and returned as the outcome of the procedure (row 12-13).

Thus, the buildControModel procedure allows the model generation process to automatically build the timeline-based specification by leveraging the knowledge about the agent. As described in [Borgo et al., 2016], every time a change occurs in the KB, a new instance of the model generation process is triggered in order to generate an updated control model of the agent. The next subsections provide some details about the (sub)procedures the model generation process relies on, and provide an example of a possible timeline-based control model that can be generated for a TM of the plant case study.

7.5.1 Building State Variables from Contexts

Algorithms 9, 10, 11 below describe the information extraction procedures that allow the model generation process to build the functional, primitive and external state variables respectively.

Building Functional Variables

Algorithm 9 describes the procedure which builds the functional state variables of the timeline-based control model. The buildFunctionalComponents procedure generates the set of state variables concerning the functional capabilities of the agent. The procedure relies on the set of capabilities the knowledge processing mechanism has inferred by applying rules 7.7 and 7.8. The procedure generates a state variable for each function of the taxonomy (see Figure 7.4) the agent can perform. Namely, given a particular function $f$ of the taxonomy, if the KB contains at least one individual for that function $f$ (i.e., if the knowledge processing mechanism has inferred at least one way for the agent to perform $f$), then a state variable $sv$ for $f$ is added to the model. The individuals of $f$ in the KB represent all the possible implementations of $f$ the agent can perform (i.e., all the capabilities of the agent with respect to $f$). Thus, for each inferred individual of $f$ the procedure adds a value to the related (functional) state variable $sv$.

The procedure first initializes the set of functional state variables of the domain (row 3). Then, it reads the taxonomy of function from the KB and, for each function, checks the available capabilities of the agent (rows 6-20). Given a function, if the KB contains at least one capability for that function, then the procedure creates a functional state variable (rows 9-11). Each capability found in the KB is modeled as a value of the related state variable (rows 12-15). The procedure ends by returning the set of obtained variables.

Building Primitive Variables

Algorithm 10 describes the procedure which builds the primitive state variables of the timeline-based control model. The buildPrimitiveComponents procedure generates the set of state variables concerning the structural components of the agent. The procedure relies on a functional interpretation of components as elements that allow the agent to perform functions. Thus, according to the inference rules 7.3, 7.4 and 7.5, the components of an agent are modeled in terms of their capabilities. The procedure adds a primitive state variable to the model for each component found in the KB. According to the inference rule 7.7, the values of these variables represent the primitive functions of the agent.

Algorithm 10 first initializes the set of primitive state variables of the domain (row 2). Then, the procedure reads the set of the inferred components from the KB (row 4). Given a component, if the KB contains at least one primitive function the agent can perform through that component, then a primitive variable is created (rows 5 -10). The values added to the variable model the capabilities of the related component. Namely, the values model the primitive functions the agent can perform by means of the considered component (rows 11-16). The procedure ends by returning the set of generated state variables.

Building External Variables

Algorithm 11 describes the procedure which builds the external state variables of the timeline-based control model. The buildExternalComponents procedure generates the set of state variables concerning the collaborators of the agent. The procedure generates a set of state variables representing the collaborators of the agent. Specifically, a state variable is created for each individual found in the KB that, according to the inference rule 7.6, has been classified as collaborator. The values of these state variables represent the operative states the collaborators may assume over time.

The procedure first initializes the set of external variables of the domain (row 2). Then, the procedure reads the set of inferred collaborators from the KB (row 4). For each collaborator found, a state variable is created (rows 5-7) and for each operative state the collaborator may assume over time, a value is added to the created variable (rows 9-14). The procedure ends by returning the set of generated variables.

7.5.2 Building Decomposition Rules from Inference Trace

When all the state variables and their values have been generated, it is necessary to build the synchronization rules of the domain in order to coordinate the temporal behavior of the state variables and achieve the desired goals. The buildSynchronizationRules procedure generates the decomposition rules by leveraging the inference trace of the KB. The inference trace represents intermediate knowledge generated by the application of inference rules. Such knowledge manages intermediate information necessary to complete the knowledge inference mechanism and therefore build the KB. For instance, besides primitive channels, the inference rule 7.7 generates cConnect properties. These properties do not represent specific information about the agent but are necessary to generate the set of complex channels, as shown in the inference rule 7.8. These properties encode functional dependencies among the components of a TM. In particular, they encode these dependencies in terms of primitive channels needed to implement complex channels.

The inferred cConnect properties can be analyzed in order to build a particular data structure, called functional graph, that correlates functional dependencies among components, primitive and complex channels. The graph is built according to the inferred cConnect properties. Thus, the possible implementations of complex channels can be found by traversing the functional graph. This set of information is necessary to build the set of synchronization rules specifying how the agent must execute complex channels. Indeed, synchronization rules are generated by analyzing the paths on the functional graph that connect the start with the end locations of complex channels. These paths can be easily expressed in terms of precedence constraints between primitive channels of the involved components.

Algorithm 12 describes the procedure for building the synchronization rules of the timeline-based domain with respect to the (complex) channel function the related TM can perform. The procedure first initializes the set of rules (row 2) and then analyzes the KB to generate the functional graph concerning channel functions (row 4). For each complex channel, the procedure extracts the available implementations from the functional graph (rows 6-9). Each implementation encodes a set of temporal constraints between the primitive channels of the agent. Thus, given a possible implementation of a complex channel, a new synchronization rule is created and added to the model (rows 10-14). The procedure ends by returning the set of generated synchronizations.

7.5.3 The Resulting Timeline-based Control Model

The procedures that have been described in the previous sections encode the model generation process which relies on the context-based characterization of the KB. According to this structure, the process generates a hierarchical domain specification modeling the complex functions of the agent in terms of the primitive functions that internal components can directly handle according to the status of the involved collaborators.

Figure 7.12 shows a (partial) example of a timeline-based control model generated for a TM composed by one cross-transfer unit only. The model provides a functional characterization of the TM according to functional, primitive and external levels cited above. The primitive state variables model the active parts of the TM that can actually perform some (primitive) functions. These state variables model the functional capabilities of elements that compose the TM. For example, the component Conveyor1 can perform the primitive channel ChannelF-Down to move a pallet between the location of component PortF and location Down of component Cross1. Similarly, the component Cross1 can perform the primitive channel ChannelDown-Up to move a pallet from the location Down to the location Up of the same component Cross1. The external state variables model the inferred collaborators that can directly interact with the considered TM. The values of these variables represent the operative states that collaborators may assume over time. Figure 7.12 shows the external state variables concerning two of four collaborators available. Specifically, the state variables model the behaviors of CollaboratorF and CollaboratorR i.e. the collaborators connected to the TM through components PortF and PortR respectively. The functional state variables model the inferred channel functions the TM can perform by combining internal (i.e. primitive) channel functions. For example, according to this interpretation, ChannelF-R can be seen as the composition of the following primitive channels:

[TABLE]

Such a composition represents a particular implementation of the ChannelF-R function. Implementations are modeled by means of synchronization rules that specify a suited set of temporal constraints (the red arrows in Figure 7.12). These temporal constraints encode also the functional dependencies between the TM and its collaborators. Indeed, CollaboratorF and CollaboratorR must be available during the execution of the ChannelF-R function. The generated timeline-based planning model provides a functional characterization of TMs of the plant where planning goals represent functions the considered TM must performpri. These functions are described in terms of the atomic operations (i.e. primitive functions) the TM is capable to perform by means of its components and its available collaborators.

7.6 The Knowledge-Based Control Loop in Action

This section reports on a set of tests on the KBCL with different TM configurations. All the different physical configurations of a TM have been considered, from zero to three cross-transfer modules. These configurations are referred to as simple, single, double and full, respectively. Each configuration also entails a different number of connected TM neighbors. Clearly, the more complex scenario is the one with the highest number of cross-transfers (the full TM) and neighbors. Also, three reconfiguration scenarios (reconf-a, reconf-b and reconf-c) have been developed considering different external events, namely an increasing number (from 1 to 3) of TM neighbors momentarily unable to exchange pallets, plus two scenarios related to internal failures (reconf-d and reconf-e) due to a cross-transfer engine failure and to a local failure on a specific port.

The experiments were carried out to evaluate the performance of the following aspects of a TM: (i) the knowledge processing mechanism; (ii) the planning model generation; (iii) the synthesis of plans to manage a set of pallet requests. The final aim is to evaluate the feasibility of the KBCL approach by showing that the performance are compatible with execution latencies of the RMS333All the experiments have been performed on a workstation endowed with an Intel Core2 Duo 2.26GHz and 8GB RAM.

Figure 7.13 shows the timings in the Setup phase for the KBCL module operation, i.e. to build the KB exploiting the classification and capability inference process (the "inference" side of Figure 7.13), and to generate the timeline-based planning specification for the TM (the "mapping" side of Figure 7.13). On the one hand, the results show that an increase in the complexity of the TM configurations does not entail a degeneration of the knowledge processing mechanism: the inference costs are almost constant (around 1.3 secs). This behavior was expected since the number of instances/relationships in the KB is rather low notwithstanding the physical configuration of the TM; thus, the performance of the inference engine deployed here is not particularly affected. On the other hand, the model generation is strongly affected spanning from 0.8 secs in the simple configuration, up to a maximum of 2.2 seconds in the full configuration. The model generation process entails a combinatorial effect on the number of instances/relationships needed to generate components and synchronizations leading to larger planning models and, thus, to higher process costs.

When a reconfiguration scenario occurs, the knowledge processing costs are negligible. Among all the considered reconfiguration cases (i.e., reconf-a-b-c-d-e), the time spent by the knowledge processing mechanism to (re)infer the enabled functionalities is just a few milliseconds. In fact, both the classification and capability inference steps are applied to a slightly changed KB and, then, minimal changes in the functionalities can be quickly inferred and represented in the new KB. For what concerns the planning model generation after a reconfiguration, each reconfiguration scenario (both external and internal) leads to a strong reduction of functionalities and, thus, the related costs are relatively small. Figure 7.14 shows the time spent to generate the planning model in the full TM configuration (i.e., the more complex configuration) is depicted and compared with respect to the time spent in the setup phase. In general, the time needed to regenerate the planning model specification is also dependent on reconfiguration scenarios but still negligible.

Finally, we evaluate the planning costs when facing both setup and reconfiguration scenarios.Figure 7.15 shows the trend of the planning time in the Setup scenario considering all the TM configurations and an increasing number of pallet requests (randomly generated), i.e., planning goals, to be fulfilled. Planning costs span from few seconds up to nearly 30 seconds when planning for 10 pallet requests within a 15 minutes horizon. In general, the more complex the planning model, the harder the plan synthesis problem. Thus, the planning costs follow the complexity of the configurations of the specific TM agent.

The experimental results show the practical feasibility of the KBCL approach in increasingly complex instances of a real-world manufacturing case study. The collected data for the initialization (or the update) of a generic agent’s KB (considering both knowledge processing and model generation) and the cost for planning synthesis have a low impact on its performance during operation. In fact, in order to face production periods of 15 minutes –and the management of 10 pallet requests– no more than 5 seconds are required by the Knowledge Manager while less than 30 seconds are required by the Planner to generate a suitable plan. Such performances are compatible with the system latency usually involved in this type of manufacturing applications.

Implementation Notes

Most of the inferences at runtime are done in the Web Ontology Language (OWL) version of the KB to exploit primarily the contextual classification and relationships. The ontology editor Protégé444http://protege.stanford.edu has been used fo KB design and testing. For runtime reasoning within the Knowledge Manager, the Ontology and RDF APIs and Inference APIs provided by the Apache Jena Software Library555http://jena.apache.org has been used.

Chapter 8 Concluding Remarks

This thesis has presented a complete characterization of the timeline-based approach ranging from a formalization of timeline-based planning to planning and execution of timelines by taking into account temporal uncertainty. After the first introductory chapters, Chapter 4 presented the formalization which defines a clear semantics of the main planning concepts like timelines, state variables, plans and goals, and taking into account the domain controllability features. Chapter 5 introduced EPSL a general framework for planning and execution with timelines which complies with the formalization and, is therefore capable of dealing with temporal uncertainty. The effectiveness of the envisaged approach has been shown by applying EPSL to real-world manufacturing scenarios within the research projects FourByThree and GECKO as described in Chapter 6 and 7 respectively. Specifically, the FourByThree project has shown how the envisaged timeline-based approach and the EPSL capabilities of dealing with temporal uncertainty both at planing and execution time, as well as the hierarchical approach, represent an effective solution for controlling a robot in scenarios where temporal uncertainty plays a relevant role, like Human-Robot Collaboration requiring a tight interaction between a controllable agent (i.e. the robot) and an uncontrollable agent (i.e. the human). The GECKO project has shown some promising results concerning the design of a flexible control architecture capable of dynamically inferring the control model by integrating knowledge reasoning techniques with timeline-based planning.

The main concern all along this work was not to design the most performing planning algorithm ever made, but rather the objective was to design and develop new and effective solutions for real-world scenarios. Thus, the key point has been to understand the features and the problems that must be considered and solved in order to effectively apply these techniques in real-world applications. For this reason, the importance of a flexible control system capable of (robustly) managing and adapting control strategies to the uncontrollable dynamics of the environment has come to light within the research projects FourByThree and GECKO.

For example, the FourByThree project shows that flexibility is needed to control the robot and dynamically adapt its behavior to the observed behavior of the human. In this case, the pursued solution consists in designing planning and execution applications capable of properly dealing with temporal uncertainty at different levels. From the planning point of view, the formal characterization of the timeline-based approach introduces the representation of the uncontrollable dynamics of the domain in shape of temporal uncertainty. Leveraging this formalization, the general hierarchy-based solving procedure of the EPSL framework has been extended by introducing temporal uncertainty in order to synthesize plans accordingly. In this way, EPSL can generate plans that have some desired properties (i.e. the pseudo-controllability property) characterizing their (temporal) robustness at execution time. From the execution point of view, once a plan has been generated it must be executed. The EPSL framework has been extended by introducing executive capabilities that rely on the same representation of the planner. Thus, EPSL can execute plans by taking into account the controllability properties of the domain and dynamically adapt the execution of the plan to the observed behavior of the environment and the related uncontrollable features.

Plan-based control architectures and also the EPSL-based control architecture typically rely on a well-defined and static model of the world. The GECKO project shows that another type of flexibility needed in real-world scenarios is the capability of dynamically adapting the control model of the plan-based controller to the specific configuration/state of the working environment and the robot (i.e. the agent to control). In this case, the pursued solution consists in designing an extended plan-based control architecture which integrates knowledge reasoning and planning. Semantic technologies introduce the capability of representing and reasoning about the state of the agent and the related working environment. Such a reasoning mechanism, embedded in the control architecture, allows for dynamically building and maintaining updated the knowledge concerning the actual functional capabilities of a particular agent. Such a knowledge is then exploited to automatically generate a control model that a plan-based controller (e.g. an EPSL-based controller) utilizes to actually plan and execute operations.

EPSL represents the main result of this work. It represents a uniform framework for planning and execution with timelines under uncertainty. In addition, the application of EPSL to the research projects FourByThree and GECKO has shown the effectiveness and the flexibility of the envisaged approach to solve real-world problems. However this is just a first step, there are several aspects to take into account in order to further improve the capabilities of the EPSL framework.

A short-term objective concerns the introduction of a flexible management of different types of resource in EPSL. In general, the objective is to enrich the types of domain element (in addition to state variables) the framework can deal with. The introduction of different types of resources (e.g. renewable resources and consumable resources) would allows EPSL and the envisaged timeline-based approach to address more realistic problems. This implies also that the solving capabilities of EPSL must be extended in order to synthesize flexible profiles for the different types of resources considered. An initial idea is to leverage temporal flexibility in order to synthesize optimistic temporal profiles of resources [Laborie, 2003, Cesta and Stella, 1997, Drabble and Tate, 1994].

Timeline-based planning systems, usually rely on a careful engineering of domain together with domain-dependent heuristics in order to control the search process. Nevertheless, there are different domain-independent heuristics that have successfully applied in classical planning showing impressive results e.g. [Hoffmann and Nebel, 2011, Blum and Furst, 1997, Helmert, 2011]. Unfortunately, the application/adaption of these techniques to timeline-based systems is neither simple nor possible. There are significant differences between the timeline-based approach and the classical approach in terms of problem representation and resolution that prevent a straightforward adaptation of these heuristics. Thus, an additional short-term objective is to investigate the design of domain-independent heuristics by borrowing ideas and concepts from classical as well as similar works in the literature e.g., [Bernardini and Smith, 2008]. The hierarchy-based technique introduced in this work represents just an initial step towards the achievement of this research objective.

Also the comparison of EPSL with other existing timeline-based systems is a relevant research objective to pursue in the near future. More in general, the objective is to compare the timeline-based approach with other temporal and hybrid planning approaches. A first contribution is represented by the work [Umbrico et al., 2016] which provides an initial comparison between EPSL and EUROPA by taking into account modeling and solving features of these two frameworks. The goal is to define a set of benchmarking problems that can be used to compare modeling and solving capabilities of EPSL with EUROPA, IxTeT and other relevant planning systems like OPTIC [Benton et al., 2012], COLIN [Coles et al., 2012], FAPE [Dvorák et al., 2014], CHIMP [Stock et al., 2015] and HATP [Lallement et al., 2014].

A medium-term objective is to further exploit temporal uncertainty at planning and execution time. With respect to planning, the objetive is to enhance the EPSL solving procedure in order to synthesize dynamically controllable plans. Pseudo-controllabiliy is a useful property but it does not provide enough information about the controllability of a plan. Indeed, pseudo-controllability is a necessary but not sufficient property for dynamically controllability. Thus, the idea is to further analyze information about the temporal uncertainty of the domain during the planning process in order to generate plans with more relevant properties characterizing their execution, i.e. dynamic controllability. With respect to execution, the objective is to integrate the synthesis and management of execution strategies [Orlandini et al., 2013, Orlandini et al., 2011], as well as validation and verification techniques [Cesta et al., 2010] in order to execute timeline-based plans in a more robust way. As it is, the executive takes execution decisions on the fly without reasoning on the overall plan and the observed behavior of the environment. The use of an execution strategy would allow EPSL to take more accurate decisions and improve the robustness of plan execution.

Finally, a long-term objective is to further investigate the integration of knowledge reasoning techniques with planning and the automatic synthesis of control models. Specifically the idea is to realize a powerful knowledge engineering tool which leverages semantic technologies in order to allow users that are not expert of planning technologies but rather expert of the domain, to use and deploy (timeline-based) planning applications. Knowledge engineering tools provide a standard and expressive interface which allows users to model the particular domain abstracting from the details of planning and problem resolution. Then, the resulting knowledge can be exploited in order to dynamically generate the planning model used to actually plan and execute operations as shown in Chapter 7.