Asynchronous Behavior Trees with Memory aimed at Aerial Vehicles with Redundancy in Flight Controller

TL;DR

This paper introduces an asynchronous behavior tree system with memory and a state synchronization algorithm to ensure mission state consistency in redundant flight control units for aerial vehicles, demonstrated in real-time HAPS simulations.

Contribution

The paper presents a novel asynchronous behavior tree framework with memory and a state synchronization method for redundant flight control systems in aerial vehicles.

Findings

System successfully maintains mission state consistency.

Proven effective in real-time HAPS simulation.

Enhances reliability of autonomous aircraft systems.

Abstract

Complex aircraft systems are becoming a target for automation. For successful operation, they require both efficient and readable mission execution system. Flight control computer (FCC) units, as well as all important subsystems, are often duplicated. Discrete nature of mission execution systems does not allow small differences in data flow among redundant FCCs which are acceptable for continuous control algorithms. Therefore, mission state consistency has to be specifically maintained. We present a novel mission execution system which includes FCC state synchronization. To achieve this result we developed a new concept of Asynchronous Behavior Tree with Memory and proposed a state synchronization algorithm. The implemented system was tested and proven to work in a real-time simulation of High Altitude Pseudo Satellite (HAPS) mission.

Tables

Table 1. TABLE I : Symmetry of three control nodes

	Sequence	Selector	Skipper
symbol	$\to$	?	$\Rightarrow$
Continue on	$𝕊$	$𝔽$	$ℝ$
Return	$ℝ$, $𝔽$	$ℝ$, $𝕊$	$𝕊$, $𝔽$

Table 2. TABLE II : Return table

		State after tick
		$ℝ$	$𝕊$	$𝔽$
State	$ℝ$	$\varnothing$	$A_R$	$A_R$
before	$𝕊$	$\varnothing$	$\varnothing$	$C_R$
tick	$𝔽$	$\varnothing$	$C_R$	$\varnothing$

Table 3. TABLE III : Call table for sequential control nodes

		tick type argument
		$A_F$	$A_R$	$C_F$	$C_R$
State	$ℝ$	$A_F$	$A_F$	$\varnothing$	$C_F$
before	$𝕊$	$A_F$	$\varnothing$	$\varnothing$	$C_F$
tick	$𝔽$	$A_F$	$\varnothing$	$\varnothing$	$C_F$

Equations

$$t:=\left\{\mathcal{G}\left(\mathcal{N},\mathcal{E}\right),\mathcal{V}\right\}$$

$$v\in\mathcal{V},scope\left(v\right)\in\left\{\begin{array}[]{l}Input,\\ Output\end{array}\right.$$

$$R=\frac{tim{e}_{classicalBT}}{tim{e}_{ABTM}}$$

Full text

I Introduction

Nowadays the complexity of unmanned aerial vehicles (UAV) and their tasks is growing rapidly. Mission execution system interacts with all robot subsystems. Therefore, the size of the mission plan increases and it introduces the need for easy-readable, efficient, and reliable control architecture (CA) with FCC redundancy support. One of the common CAs is the Behavior Tree (BT), originated from game industry[1]. They have been widely used because of their modularity and reactivity properties. Behavior tree was suggested as a CA for unmanned aerial vehicles in[2], [3]. There are several studies on BT applications in other areas of robotics, e.g. surgical robot[4] and robotic assistants[5]. Scientists touched many aspects of BT, such as a generalization of other CA[6], mathematical model formulation[7], automated BT construction [8], and others. The compherensive introduction to synchronous BT in robotics could be found in[9].

One of the recent topics in this area is asynchronous (or event-driven) BT, which are deviant from classical synchronous BT formulation. It was shown that asynchronous BT outperforms synchronous in continuous-time mission simulation[10]. Being flexible and reactive, behavior trees frequently require sort of inner memory, e.g., for a way-point missions.

To solve this issue, the Reset and Latch node combination was suggested [11] as an alternative to control nodes with memory[12]. A similar interaction between nodes could be done by a “blackboard” - a key-value table which is accessible by all tree nodes and sometimes other software modules. Although some authors argue that this approach “does not lend itself well to encapsulation and, as a result, frustrates subtree reuse”[13], blackboards are widely used in both behavior tree frameworks for video game industry applications[1] and in robotics [4],[5].

This work is devoted to designing a MES for UAV application. Our target platform is HAPS (Fig. 1). HAPS has to be autonomous to achieve continuous flight for several days or months. To achieve the required level of hardware reliability, HAPS is equipped with redundant flight control computers (FCC). However, this redundancy should be additionally supported on the software level. Missing the data samples or different order of received messages would break consistency of mission execution state on redundant FCC units. This issue might lead to unstable behavior of the HAPS, e.g., attempting execution of different commands. We developed statespace synchronization procedure to maintain state consistency on all FCCs. We took an advantage of asynchronous BT propagation and formulated a new four tick propagation types to satisfy the goal for continous time simulation efficiency.

In our BT architecture, interaction both with the other modules and between nodes is done completely by the blackboard, called hereafter memory.

We introduce a new Skipper control node which is symmetric to the well-described Selector and Sequence nodes.

The paper structured as follows: in Section II we describe changes in semantics and new nodes. Section III is devoted to explaining the asynchronous mechanism of call propagation in the tree from receiving a new information sample to sending out changes. In Section IV we describe how to support hardware redundancy on FCC with the developed framework. Section V denoted to implementation and conducted tests. In Section VI there are a conclusion and future work plans.

II Changes in Behavior Tree Formulation

Semantics

it is hard to name certain BT formulation as a canonical because almost every work on Behavior Tree slightly varies in semantics (see comparison[12]). The BT definition consists of a graph of nodes ($\mathcal{N}$) and edges ($\mathcal{E}$) which is a directed tree and a memory, which is a key-value dictionary ($\mathcal{V}$):

[TABLE]

Each node in behavior tree has a state: $Running,Success$ or $Failure$. In the paper they are shorten respectively as $\mathbb{R}$, $\mathbb{S}$, $\mathbb{F}$. Nodes are separated into two groups: leaf and control. Control nodes define the inner logic of the executor while leaf nodes are aimed to interact with the environment. Each node has a tick function, which updates its state due to the definition of the node.

Significant changes in our BT semantics comparing to other works are:

• •

Action and Condition nodes rely on external calls to corresponding middleware $\rightarrow$ Action and Condition nodes are functions over inner memory.

• •

Conditions nodes invoke binary ($\mathbb{S}$, $\mathbb{F}$) functions, Action nodes might be in ($\mathbb{R}$) state $\rightarrow$ Conditions might return one of ($\mathbb{R}$, $\mathbb{S}$, $\mathbb{F}$) states, Actions always return $\mathbb{S}$.

• •

The tick function is periodically applied to the root node, performing whole tree traversal $\rightarrow$ Only start of the execution is done by a single tick applied to the root node. The rest is executed in an asynchronous callback function.

Memory

in our approach BT interacts with outer scope by assigning and reading the memory variables. For simplicity reasons, we assume that all variables have floating-point values and string keys. Each variable has a Scope property:

[TABLE]

Only $Output$ variables would be added to return of callback if some Action changes their value. We also remember last result $state(\mathcal{N})\subset\mathcal{V}$ of node evaluation. This is done to clearly define propagation rules.

Leaf nodes

as it is widely accepted [1],[12], there are two leaf executable nodes which are able to interact with the outer scope (through $\mathcal{V}$): Action and Condition. In a memory-based tree, semantics of leaf nodes has to be changed. Action node modifies $\mathcal{V}$ and always returns $\mathbb{S}$, as variables assignment could not fail or last for a significant time. Condition node does not modify the variables and returns a state $\in\left\{\mathbb{R},\mathbb{S},\mathbb{F}\right\}$. On a change of at least one variable from this subset a condition should be reevaluated.

Transferring an ability to have $\mathbb{R}$ state to Condition node could be additionally motivated from a mission design point of view. Let us assume we need to handle multiple responses for one action (Fig. 3). Now we clearly separate the responses implemented by conditions from call to the external module made action. In contrast, previous works suggested to add extra states for Action nodes [11].

Control nodes

are Sequence, Selector, Parallel, and Skipper. First three are extensively discussed before[12], while Skipper is a new node type. However, it is a “sibling” of Sequence and Selector in terms of evaluation (see Table I)

They all tick their children and check the state sequentially from left to right. If the state of the child belongs to the “Return” subset, control node return the state immediately. In the other case, they continue evaluation and return “Continue on” state. These three control nodes have similar evaluate functions, see Algorithm 2. There are several possible applications of a new control node. One can treat $\mathbb{R}$ state of a Condition node as undefined or unknown and implement passing the decision making to the next Condition node (Fig. 4).

Latch concept

another interesting application of Skipper node is an implementation of Latch concept (Fig. 5). Latch remembers the first $\mathbb{S}$ or $\mathbb{F}$ returned state of subtree and do not evaluate the subtree until reseted [11].

Supplementary Reset node could be implemented just by a single Action node that assigns $mem:=0$ at some place of the tree.

III Asynchronous Tick Propagation

As a mission executor, BT has to acquire input data, respond to this data, modify its own state and sometimes call actions activating changes in corresponding modules of the autonomous system. Incoming data should first be converted to changes in $\mathcal{V}$. Than, the response to any change to input is done by a callback function. An argument of the function is a dictionary (key - name of variable $v\in\mathcal{V}$, value - new value of $v$). If variables from $Output$ subset were modified through the callback execution, it would be added to the returned dictionary (see also Fig. 6). The callback function consists of four main parts:

Apply the changes to the memory. 2. 2.

Re-evaluate conditions, that depends on changed variables. Add conditions which changed their state to the ordered queue $nodes\_to\_tick$. 3. 3.

While $nodes\_to\_tick$ queue is not empty, take the first node, call its tick method and by returned $state$ and $tick\_type$ decide if we add its parent to the queue or not; also add changed conditions, if any. 4. 4.

If there are no nodes left to be ticked, return the changes in $Output$ subset of memory.

Tick types

in synchronous BT tick was always propagated in a top-down or Fall manner. In the asynchronous tree we have to define rules for bottom-up or Rise propagation. In order to do this, we added a $tick\_type$ parameter for the tick function. Moreover, tick function returns the state of the node and a tick type which should be applied to the parent. For efficient propagation we add another category - ticks might be either Activating or Checking. The second one is used when we do not need to recursively evaluate subtrees of a control node and could look only on their last state updates. Thus, four tick types are Activating Fall ($A_{F}$), Activating Rise ($A_{R}$), Checking Fall ($C_{F}$), and Checking Rise ($C_{R}$). $A_{F}$ represents the propagation manner that is happening in synchronous trees (parent recursively ticks children, we activate actions). $A_{R}$ is rising of $A_{F}$ (bottom up, child adds parent to $nodes\_to\_tick$ queue). $A_{R}$ happens when state changes from Running to Success/Failure implying continuing the paused $A_{F}$ tick. $C_{R}$ represents bottom-up propagation of state checking (without activation of any action node). $C_{F}$ refers to look-up for children state (without further fall). $N_{T}$ or $\emptyset$ symbol is used when no tick is required.

The definition of tick function is the same for each node (Alg. 3). The evaluate function is unique for each node type. Evaluate accepts $tick\_type$ argument and returns the resulting state of the node evaluation (Alg. 2). Definition of both tick and evaluate functions is discussed later after explaining tick propagation rules.

Order of nodes in the tree

undefined order of bottom-up propagation of multiple conditions’ changes could lead to different output results. Thus, we have to clarify the order of nodes in $nodes\_to\_tick$ priority queue. One needs to ensure that the parent would not be evaluated before the child and children would be evaluated from left to right. These rules are exactly the description of Kleene-Brouwer order. To compare, each node holds an order, which is an array of integers. The root node has an order containing one number $\{0\}$. To make a child order, we append its position in children list to the order array of the parent. See also a behavior tree example with node order (Fig. 5).

Propagation rules

the propagation rules could be clearly defined by two tables for each type of a node. First table – return tick table (see Table II) – is the same for all node types and illustrates which tick type would be applied to the parent node in the case of bottom-up propagation. The returned tick type depends only on states before and after node evaluation.

Hence the diagonal elements of the table are $\emptyset$, meaning there would be no bottom-up propagation in case the state of the node was not changed. The same holds true if the state changed to $\mathbb{R}$ from either $\mathbb{S}$ or $\mathbb{F}$. This change could be treated as activating from the bottom to the top. We forbid such changes because task activation is naturally triggered from the top to the bottom. However, altering elements in a table is a flexible way to change rules of tick propagation. Success to Failure changes are $C_{R}$ because they do not imply any local task resumption. Finally, $\mathbb{R}$ to $\mathbb{S}$ or $\mathbb{F}$ changes are $A_{R}$.

Table III – call table – illustrates how the control node should evaluate the children. The tick type passed as argument to evaluate function depends on a current state and a tick type passed as argument to tick function. Different control nodes might have different evaluation tick tables. Let’s thoroughly describe the content of the evaluation tick table for Sequence. $A_{F}$ column represents the activation process. All the nodes are recursively evaluated, encountered actions are called. The rest is specific for asynchronous BT. The second column in $\mathbb{S}$ and $\mathbb{F}$ rows shows the ban for bottom-up activation propagation. Last 2 columns mean that for $\mathbb{S}\leftrightarrow\mathbb{F}$ transition tree does not resume any activation process. As for the Parallel node we can change $A_{F}$ to $C_{F}$ in the $A_{R}$ column. All subtrees are activated through the initial $A_{F}$ tick, therefore, the node does not need to activate them later.

Tick and evaluate functions

tick function is the same for every node type (see the definition in Alg. 3). Its main purpose is to obey propagation rules defined above and call the evaluate function with proper tick_type argument.

Synchronous tree inside asynchronous

Synchronous BT could be implemented as a subtree inside asynchronous having a periodical time event (see Fig. 7)

IV Execution With Redundancy on Flight Control Computers

We need to synchronize the states of FCC mission execution systems. In our system, any changes are transferred to the outer scope only after calling callback function. Assume we started the same trees on all FCCs. Then, result of callback depends only on variable values $var(\mathcal{V})$ and node states $state(\mathcal{N})\subset\mathcal{V}$. Thus, if all three FCCs have the same $\mathcal{V}$ before callback execution, they would have the same output. That means we need to be sure in variable values constistency before each callback. In fact, we have to synchronize variables only if some condition has changed. If no condition changed, there would be no tick propagation started and no output changes produced. Condition change events usually lead to the execution of some action and, thus, are much more rare than incoming sample events. The developed memory and asynchrony features of BT helped to drastically reduce the amount of synchronization events.

This logic was implemented by modifying callback function and adding an extra independent BT sync_tree for synchronization. Fig. 8 shows the concept of this complementary BT. It consists of 5 main branches:

Condition that activates further execution. 2. 2.

Latched Action node that sends a hash of $\mathcal{V}$ to other FCCs. 3. 3.

Subtree that waits for the hashes from other FCCs and chooses a new master FCC if the current master is not responding. 4. 4.

Subtree that sends changes in $\mathcal{V}$ from the master FCC to slaves if their hashes differ. 5. 5.

Action that returns all variables to their initial values.

Real synchronization procedure and corresponding behavior tree might be even more complicated (e.g. timeout conditions for any communication with other FCCs).

V Implementation and Tests

C++ framework

we developed a C++ framework with the implementation of described Asynchronous Behavior Tree with Memory (ABTM) concept. It includes tree construction and execution. For leaf nodes one can use C++ functions or construct nodes from simple expressions (such as $velocity==0$ or $x=1$). The framework was developed with thread safety taken into account.

Real-time simulation

in order to verify our concepts, we also implemented synchronization procedure following the guideline from Section IV. We connected our framework to the real-time HAPS mission simulation. It imitates the behavior of all other aircraft modules, including high-frequency data flow from positioning and all control modules. To prove our system working we started three independent mission executors. Only one current master executor was allowed to send output to other modules. All FCCc are shown to maintain the same state in long-time runs. Another test was a crash imitation by simultaneously stopping any of the executors at random time. In this case, the remaining one or two executors chose a new master and the mission continued correctly.

Efficiency evaluation

we compared the computation times for asynchronous execution (Section III) and classical top down tree traversal (applying $A_{R}$ to the root). To eliminate the difference between certain BT structures we created 200 random tree structures with height in the range of $3..5$. Number of each control node children was in the range of $3..7$. We averaged the results for different complexities of conditions. ABTM was proven to be faster in all of the cases. We plotted a ratio between classical and asynchronous time execution (Fig. 9).

[TABLE]

Dense case illustrates the situation when every incoming sample changes at least one condition and starts tree traversal for ABTM. Sparse case is the opposite to the dense – every sample does not trigger any condition. ABTM outperforms classical approach in all cases with ratio $R\in[10,70]$ for trees with $300$ nodes. This feature speeds up the continous time simulation.

In addition, ABTM reduces the amount of synchronization call in case of redundant execution (Section IV). However, the difference highly varies with application. If the robot executes an action every $3$ seconds and classical BT runs with $20~{}Hz$ update rate, then the amount of synchronization calls would be reduced by $3\cdot 20=60$ times.

VI Conclusions

We formulated a new Asynchronous behavior tree with memory approach. It includes a description of the memory layer, rules of tick propagation, control and leaf nodes definition. Overall, the developed BT functionality is richer and more flexible than previous definitions. Lower computational costs are beneficial for continous time simulations. In addition, asynchronous approach helped us decrease the amount of synchronization calls in execution on redundant FCCs.

Our approach also allowed to make a standalone C++ library. An ability for automated testing the BT logic outside the robot environment in a simple simulation mode (e.g. applying necessary input changes and checking the outputs) is also a benefit.

Our changes in BT semantics, especially memory layer, simplified the development of the synchronization algorithm for multiple FCCs. We demonstrated the possibility to use our framework on robotic systems with redundancy on FCC units and tested it in the simulation.

Our future plans include flight experiments of unmanned HAPS missions using a developed framework with triple redundancy support.