Priority Inheritance with Backtracking for Iterative Multi-agent Path Finding

TL;DR

This paper introduces PIBT, a scalable sub-optimal algorithm for iterative multi-agent pathfinding that guarantees agent delivery in certain graph environments and performs well in large, real-world scenarios.

Contribution

PIBT is a novel priority inheritance with backtracking algorithm that efficiently solves large-scale, iterative MAPF problems with guaranteed agent delivery in specific graph conditions.

Findings

PIBT guarantees all agents reach their destinations in finite time on certain graphs.

PIBT performs efficiently with hundreds of agents, providing acceptable solutions rapidly.

PIBT outperforms existing methods in runtime and solution quality in warehouse package conveyance scenarios.

Abstract

In the Multi-Agent Path Finding (MAPF) problem, a set of agents moving on a graph must reach their own respective destinations without inter-agent collisions. In practical MAPF applications such as navigation in automated warehouses, where occasionally there are hundreds or more agents, MAPF must be solved iteratively online on a lifelong basis. Such scenarios rule out simple adaptations of offline compute-intensive optimal approaches; and scalable sub-optimal algorithms are hence appealing for such settings. Ideal algorithms are scalable, applicable to iterative scenarios, and output plausible solutions in predictable computation time. For the aforementioned purpose, this study presents Priority Inheritance with Backtracking (PIBT), a novel sub-optimal algorithm to solve MAPF iteratively. PIBT relies on an adaptive prioritization scheme to focus on the adjacent movements of multiple agents; hence it can be applied to several domains. We prove that, regardless of their number, all agents are guaranteed to reach their destination within finite time when the environment is a graph such that all pairs of adjacent nodes belong to a simple cycle (e.g., biconnected). Experimental results covering various scenarios, including a demonstration with real robots, reveal the benefits of the proposed method. Even with hundreds of agents, PIBT yields acceptable solutions almost immediately and can solve large instances that other established MAPF methods cannot. In addition, PIBT outperforms an existing approach on an iterative scenario of conveying packages in an automated warehouse in both runtime and solution quality.

Tables

Table 1. Table 2: PIBT in extremely dense situation. The used map is empty-8-8 . 25 instances were prepared for each | A | 𝐴 |A| . Scores of “sum-of-costs” and “makespan” are upper bounds of sub-optimality in the same way as Fig.  6 .

$\left|A\right|$	success rate	runtime (ms)	sum-of-costs	makespan
40	0.96	0.21	3.15	3.46
50	0.84	1.43	7.38	6.94
60	1.00	2.16	12.25	7.86
64	1.00	3.16	21.55	10.01

Table 2. Table 3: Effect of tie-break strategy. “normal” is PIBT and “random” is PIBT without the rule-2 tie-break. The used map was den520d . For each | A | 𝐴 |A| , 25 random scenarios were prepared, not equivalent to those in Sec.  5.1 . The scores of sum-of-costs and makespan are average of upper bounds of sub-optimality with 95% confidence intervals, based on instances that were solved by both.

	success rate		sum-of-costs		makespan
$\left|A\right|$	normal	random	normal	random	normal	random
100	1.00	1.00	1.04 (1.04, 1.05)	1.08 (1.07, 1.09)	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)
300	1.00	1.00	1.10 (1.09, 1.10)	1.16 (1.15, 1.16)	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)
500	0.96	1.00	1.15 (1.14, 1.15)	1.22 (1.22, 1.23)	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)
700	0.96	0.96	1.20 (1.20, 1.20)	1.28 (1.27, 1.28)	1.00 (1.00, 1.00)	1.00 (1.00, 1.00)
900	0.88	0.96	1.25 (1.24, 1.25)	1.33 (1.32, 1.34)	1.00 (1.00, 1.00)	1.00 (1.00, 1.01)

Full text

For the aforementioned purpose, this study presents Priority Inheritance with Backtracking (PIBT), a novel sub-optimal algorithm to solve MAPF iteratively. PIBT relies on an adaptive prioritization scheme to focus on the adjacent movements of multiple agents; hence it can be applied to several domains. We prove that, regardless of their number, all agents are guaranteed to reach their destination within finite time when the environment is a graph such that all pairs of adjacent nodes belong to a simple cycle (e.g., biconnected). Experimental results covering various scenarios, including a demonstration with real robots, reveal the benefits of the proposed method. Even with hundreds of agents, PIBT yields acceptable solutions almost immediately and can solve large instances that other established MAPF methods cannot. In addition, PIBT outperforms an existing approach on an iterative scenario of conveying packages in an automated warehouse in both runtime and solution quality.

1 Introduction

In systems with multiple moving agents, valid paths must be provided to ensure that agents reach their destinations smoothly without any collisions, while minimizing excess travel time. This problem is embodied in Multi-Agent Path Finding (MAPF) [1], which aims to find a set of collision-free paths on graphs. Currently, MAPF is receiving considerable attention because it is highly applicable in problems such as intersection management [2], automated warehouse [3], games [4], airport surface operation [5], and parking [6]. However, the optimization of this problem is known to be computationally intractable  [7, 8, 9, 10] because of the exponential growth of the search space as the number of agents increases.

Prior research on MAPF primarily focuses on solving a “one-shot” version of the problem, where agents reach their respective goals from their initial positions only once. In practical applications such as conveying packages in a warehouse with hundreds or more robots [3], MAPF must be solved online and iteratively on a lifelong basis. That is, whenever an agent reaches a goal, it receives a new one, and the plan, a set of paths, is updated in real-time. Such scenarios rule out simple adaptations of offline and compute-intensive optimal approaches because, despite state-of-the-art optimal algorithms [11, 12], it is challenging to find solutions for a few hundred agents within a realistic timeframe. For this purpose, an attractive possibility is to design scalable sub-optimal algorithms that output plausible solutions within a predictable computational time, which also works in iterative situations. Such algorithms are also promising for real-time planning where deliberation time is limited because it enables iterative refinement for known MAPF solutions [13].

To this end, a novel sub-optimal algorithm named Priority Inheritance with Backtracking (PIBT) is proposed to solve MAPF iteratively. Theoretically, PIBT ensures reachability, that is, every agent always reaches its destination within a finite time when the environment is a graph such that all pairs of adjacent nodes belong to a simple cycle (e.g., biconnected). PIBT also has low-cost time complexity. It repeats one-timestep planning until termination (e.g., until all agents have reached their destinations). For each timestep, PIBT requires the time complexity $O(|A|\cdot(\Delta(G)+F+\log{|A|}))$, where $|A|$ is the number of agents, $\Delta(G)$ is a diameter of a graph $G$, and $F$ is the time required to evaluate the distance from one vertex to the destination. Note that $F$ can be constant with preprocessing. As a result, PIBT is expected to work in a short runtime, even in the case of massive problems.

From the empirical end, PIBT immediately returns acceptable solutions despite hundreds or more agents in running times that are orders of magnitudes faster than other established MAPF approaches, such as sub-optimal priority-based [4], rule-based [14], and search-based [15] algorithms, as well as state-of-the-art optimal search-based [16] and compiling-based [17] algorithms. For instance, using an ordinary laptop, we observe that PIBT solves one-shot MAPF in a large grid map ($530\times 481$, number of vertices: $43,151$) from [1] with $1,000$ agents in at most 5 seconds, while keeping sub-optimality below $1.5$ on average. Furthermore, PIBT outperforms an existing approach for both runtime and solution quality on an iterative scenario of conveying packages in an automated warehouse, called Multi-Agent Pickup and Delivery (MAPD) [18].

Reachability does not ensure that all agents are on their goals simultaneously; hence, PIBT is incomplete for conventional MAPF. However, agents often do not need to stay at their goals in real-life applications such as lifelong delivery tasks. Reachability is convenient in such scenarios. Indeed, this study introduces the application of PIBT to a lifelong variant of MAPF (i.e., MAPD [18]) while ensuring completeness.

The mechanism of PIBT is simple and easy to implement. The algorithm focuses on the adjacent movements of multiple agents based on prioritized planning [19] in a unit-length time window. Priority inheritance is a well-known approach for dealing effectively with priority inversion in real-time systems [20] and is applied here to path adjustment. When a low-priority agent–X impedes the movement of a higher-priority agent–Y, agent–X temporarily inherits the higher-priority of agent–Y. To avoid agents getting stuck waiting, priority inheritance is executed in combination with a backtracking protocol. Because this mechanism only requires local interactions between agents, PIBT has a high potential for decentralized implementations.

In summary, the main contributions are two-fold:

We propose the PIBT algorithm, which solves MAPF iteratively and guarantees reachability. 2. 2.

We evaluate the algorithm in various environments to assess its practicality. In particular, experimental results analyzing various scenarios, including robot demonstration, confirm the adequacy of the algorithm both in finding paths in large environments with many agents as well as in conveying packages in an automated warehouse.

All external materials (e.g., code and a movie) are available at https://kei18.github.io/pibt2. The techniques presented here provide practical insight into online and lifelong scenarios involving a swarm of agents.

The rest of the paper is organized as follows. Section 2 reviews existing MAPF studies. Section 3 defines MAPF and its variant called MAPD, a problem considering operations in automated warehouses. Then, Section 4 presents the PIBT algorithm along with a theoretical analysis of its reachability and time complexity. In Section 5, we empirically evaluate PIBT using many simulated scenarios and a demonstration on real robots. Finally, Section 6 concludes the paper and discusses future directions.

Difference from Conference Versions

The PIBT algorithm was initially presented in our preliminary paper [21]. PIBT+ (Sec. 4.4.1) was initially mentioned in [13]; however, theoretical analyses were not provided; we remedy this shortfall here. This paper, in addition to improving the description and presentation of PIBT, has the following major differences from the previous versions: (1) Exhaustive experiments were conducted on one-shot MAPF using various fields and solvers. As a result, this paper thoroughly clarifies the advantages and disadvantages of PIBT compared to other de facto approaches. The implementation was also significantly improved, with the running time decreased by orders-of-magnitudes than that of the conference version [21]. This is mainly due to distance evaluation, as explained in Section 4.4.1. (2) Further theoretical analyses were conducted on PIBT, for instance, on reachability considering rotation conflicts. (3) A stress test for the number of agents and the robot demonstration of PIBT in an iterative scenario were added.

2 Related Work

This section describes the basis of MAPF, summarizes the relationship between PIBT and other existing MAPF algorithms, and presents lifelong and online situations apart from conventional MAPF studies.

2.1 Basis of Multi-Agent Path Finding (MAPF)

The MAPF [1] assigns each agent on a graph a collision-free path to its destination. Time is discrete; for each timestep, each agent performs atomic action in a synchronized manner such that it either moves to an adjacent node or stays in its current location. MAPF is known to be an NP-hard problem according to various optimization criteria [22, 7], which holds for restricting fields in planar graphs [9], in grid structures [10], and in approximating within any constant factor less than 4/3 for the last arrival time metric [8]. Limiting the discussion to sub-optimal solutions on undirected graphs, there exists a procedure of $O(|V|^{3})$ operations [23] for the compatibility of the pebble motion problem (a generalization of the sliding tile puzzle), where $|V|$ is the number of vertices. In contrast, finding solutions on directed graphs itself is NP-complete [24]. The case of strongly biconnected digraphs is special, where there is a polynomial-time procedure [25] to solve MAPF for such graphs. The proposed method, PIBT, is neither complete nor optimal for MAPF. However, PIBT works with both directed and undirected graphs.

2.2 Algorithms

To date, the numerous solvers developed for MAPF can be categorized into optimal or sub-optimal, complete or incomplete. Here, an algorithm is complete when it ensures to return a set of collision-free paths such that the end of each path is the destination of each agent. If no such set of paths exists, complete algorithms must report its absence. Another categorization is solving style, by which algorithms are classified into the following five classes:

• •

Search-based approaches search feasible solutions in a coupled manner among agents. Examples are $\text{A}^{\ast}$ with operator decomposition [26], increasing cost tree search [27], enhanced partial expansion $\text{A}^{\ast}$ [28], conflict-based search (CBS) [16], and M∗ [29]. These algorithms are complete and can solve MAPF optimally. Bounded sub-optimal versions [30, 15], which are complete, have also been developed to solve large instances.

• •

Compiling-based approaches reduce MAPF to well-known problems, such as, SAT [31, 32, 33], integer linear programming [34, 17], and answer set programming [35, 36]. Those algorithms are complete and optimal.

• •

Prioritized planning [19] approaches sequentially plan individual paths for agents in a decoupled manner; in general, these algorithms are neither complete nor optimal.

• •

Rule-based approaches make agents move step-by-step following ad-hoc rules. Examples include a graph abstraction approach [37], [38] for spanning trees, BIBOX [39] for biconnected graphs, and push and swap/rotate [14, 40] for arbitrary graphs. The rule-based approaches are sub-optimal but often ensure completeness for certain problem instances.

• •

Learning-based approaches use machine learning techniques and imitate good behavior of agents for expert data. Examples include reinforcement learning [41, 42] and graph neural networks [43, 44]. They are suboptimal and incomplete.

See [45, 46] for comprehensive reviews. Despite state-of-the-art search-based or compiling-based optimal approaches, planning with a few hundred agents within a reasonable time is challenging [12, 11]. Learning-based approaches scale well in certain cases; however, they do not have any theoretical guarantees. Therefore, for the target scenarios considered in this study (i.e., online and lifelong scenarios with hundreds or more agents), prioritized planning and rule-based approaches are promising methods. Indeed, PIBT combines these two classes. The following reviews focus on prioritized planning and rule-based approaches. Note that PIBT is “incomplete” because it ensures that all agents eventually reach their destinations; however, they may not be there simultaneously. This property is designed for lifelong situations and significantly differs from the theoretical properties provided by existing algorithms for one-shot scenarios.

Prioritized planning [19] is neither complete nor optimal; however, it is a computationally inexpensive approach to MAPF. It is based on the concept that agents sequentially plan paths according to their unique priorities while avoiding conflicts with previously planned paths. Because of its computational efficiency and simplicity, prioritized planning is highly favorable in practice and is often used as part of MAPF solvers. For instance, Wang and Botea [47] combined techniques of prioritized planning for the way the blanks moved around in sliding tile puzzles and proposed a complete algorithm for a specific class of instances. Velagapudi et al. [48] proposed decentralized implementations of prioritized planning that divide planning into several iterations. Variants of prioritized planning have also been studied, e.g., multi-robot decoupled planning that first computes individual plans neglecting collisions, and then replans individual paths while incorporating collision costs gradually [49]. We acknowledge that this scheme has similarities to PIBT, albeit the applied problem is different and the approach has no theoretical guarantee. The well-known algorithm of prioritized planning for MAPF is hierarchical cooperative $\text{A}^{\ast}$ (HC$\text{A}^{\ast}$) [4]. The “windowed” version of HC$\text{A}^{\ast}$ is known as windowed HC$\text{A}^{\ast}$ (WHC$\text{A}^{\ast}$) [4], which repeatedly plans fixed-length (windowed) paths for all agents. PIBT can be considered as using WHC$\text{A}^{\ast}$ with a unit-length window sizes.

Because priority ordering is crucial for prioritized planning, several studies addressed this issue. The negotiation process between agents regarding the ordering was studied in [50]; this process solves conflicts by having involved agents try all priority orderings and deals with congestion by limiting negotiation to at most three agents while letting others wait. A similar approach is taken in [51] where the winner determines the prioritization scheme. There are some heuristic approaches for offline planning to find a reasonable ordering, for instance, adjusting the ordering by simple hill-climbing [52], or using distances between initial locations and destinations [53]. There is a sufficient condition that the sequential collision-free solution can always be constructed regardless of priority orders, called well-formed instances [54], such that for each pair of start and goal, a path exists that traverses no other starts and goals. However, well-formed instances are difficult to realize in dense situations. A recent theoretical analysis for prioritized planning [55] identifies instances that fail for any order of static priorities, which provides a strong case for planning based on dynamic priorities, such as the approach taken with PIBT. The paper also presents a variant of prioritized planning that searches for good static priorities using techniques from CBS; however, the method has no theoretical guarantee.

Push and swap/rotate (PS) [14, 40], which partly influenced the proposed PIBT, is a sub-optimal rule-based approach for arbitrary graphs. It relies on two primitives: the “push” operation to move an agent toward its goal and the “swap” operation to allow two agents to swap locations without altering the configuration of other agents. These approaches, however, only allow a single agent or a pair of agents to move at a time. Some studies enhanced PS, for instance, parallel push and swap [56] by enabling all agents to move simultaneously, or push–swap–wait [57] by taking a decentralized approach in narrow passages. Wei et al. [58] proposed a decentralized approach that partly uses the push and swap technique to avoid deadlocks. DisCoF [59], a decentralized method for MAPF, also uses swap operation to ensure completeness. There is an algorithm similar to PS, called tree-based agent swapping strategy (TASS) [60], targeting trees. PIBT can be regarded as a combination of safe “push” operations because of the backtracking protocol and dynamic priorities. Note that PIBT does not require the operational equivalent of “swap.”

2.3 Lifelong and Online Situations

Conventional MAPF focuses on solving a *“one-shot”*111A “one-shot” problem is a problem that has clearly defined inputs and must be solved only once. This is in contrast to problems that must be solved repeatedly for a stream of inputs.

and offline version of the problem, that is, ensuring that agents reach their goal from their initial position just once. Several variants of the MAPF problem [18, 61, 62] aim to address the needs of more realistic scenarios, including lifelong and online operations. Among them, Multi-Agent Pickup and Delivery (MAPD) [18] abstracts real scenarios such as an automated warehouse and has both task allocation and path planning. Agents are assigned to a task from a stream of delivery tasks and must consecutively visit both a pickup and a delivery location. The study proposes two decoupled algorithms based on HC$\text{A}^{\ast}$ for MAPD, called token passing and token passing with task swap, respectively. These algorithms can be decentralized but require a certain amount of non-task endpoints where agents do not block other agents’ motions. Online MAPF [61] addresses situations with a dynamic group of agents, i.e., new agents might appear at runtime, or agents might disappear when they reach their destinations. We do not address this problem; however, the adaptation of PIBT for this is straightforward because PIBT essentially repeats one-timestep planning.

3 Problem Definition

This study focuses on two problems: conventional one-shot MAPF problem [1] and one of its variants, the MAPD problem [18] for lifelong and online scenarios. We formulate both problems in this section. Note that the adaptation of PIBT to other lifelong MAPF variants [61, 62] is straightforward.

3.1 Multi-Agent Path Finding (MAPF)

An instance of MAPF is given by a directed graph $G=(V,E)$, a set of agents $A=\{a_{1},\ldots,a_{n}\}$ with $|A|\leq|V|$, an injective initial state function $s:A\mapsto V$, and an injective goal state function $g:A\mapsto V$. To simplify the notation, let $s_{i}$ and $g_{i}$ denote $s(a_{i})$ and $g(a_{i})$, respectively. Considering practical situations, a graph $G$ must satisfy the following two properties: (1) simple, i.e., neither (self) loops nor multiple edges (between the same vertices) are present. (2) strongly-connected, i.e., every node is reachable from every other node. This includes all simple undirected graphs that are connected.

Let $\pi_{i}[t]\in V$ denote the location of an agent $a_{i}$ at discrete time $t\in\mathbb{N}$. At each timestep $t$, $a_{i}$ can move to an adjacent node, or stay at its current location, i.e., $\pi_{i}[t+1]\in\mathit{Neigh}(\pi_{i}[t])\cup\{\pi_{i}[t]\}$, where $\mathit{Neigh}(v)$ is the set of nodes neighboring node $v$ in graph $G$. Agents must avoid two types of conflicts: (1) vertex conflict: $\exists i,j,\;i\neq j,\;\pi_{i}[t]=\pi_{j}[t]$ (when agents share the same location), and, (2) swap conflict: $\exists i,j,\;i\neq j,\;\pi_{i}[t]=\pi_{j}[t+1]\land\pi_{i}[t+1]=\pi_{j}[t]$ (when two agents swap their occupying vertices). We refer to a set of paths as conflict-free when those two conditions are satisfied.

A solution to MAPF is a set of conflict-free paths $\{\pi_{1},\ldots,\pi_{n}\}$ such that all agents reach their goals at a certain timestep $T$. More precisely, the problem assigns a path $\pi_{i}=(\pi_{i}[0],\pi_{i}[1],\dots,\pi_{i}[T])$ to each agent such that $\pi_{i}[0]=s_{i}$ and $\pi_{i}[T]=g_{i}$. Note that all agents must be at their goals at the same timestep. Hence, $T$ is common among agents.

Two different metrics are commonly used to evaluate the quality of MAPF solutions: (1) sum-of-costs: $\sum_{a_{i}\in A}T_{i}$ where $T_{i}$ is the earliest timestep such that $\pi_{i}[T_{i}]=g_{i},\dots,\pi_{i}[T]=g_{i},T_{i}\leq T$, (2) makespan, i.e., the value of $T$.

3.2 Multi-Agent Pickup and Delivery (MAPD)

Similar to MAPF, a graph $G=(V,E)$ and a set of agents $A=\{a_{1},\ldots,a_{n}\}$ with $|A|\leq|V|$, and an injective initial state function $s:A\mapsto V$ (for simplicity, $s(a_{i})$ is denoted as $s_{i}$), are given.

Consider a stream of tasks $\Gamma=\{\tau_{1},\tau_{2},\ldots\}$. A task $\tau_{j}$ is defined as a tuple $(v_{\textit{pickup}},v_{\textit{delivery}})$, where $v_{\textit{pickup}},v_{\textit{delivery}}\in V$. MAPD includes situations in which tasks are added to $\Gamma$ as time progresses. In other words, it is not assumed that all tasks are known initially. An agent is free when it has no assigned task. A task $\tau_{j}\in\Gamma$ can only be assigned to free agents and is assigned to at most one agent. When $\tau_{j}$ is assigned to $a_{i}$, $a_{i}$ has to visit $v_{\textit{pickup}}$ and $v_{\textit{delivery}}$ in order. When the two nodes have been visited by $a_{i}$, $\tau_{j}$ is completed, and $a_{i}$ becomes free again. A service time of $\tau_{j}$ is defined as the time interval from the generation of $\tau_{j}$ to its completion. Similar to MAPF, time is discretized. At each timestep, each agent can move to an adjacent node or stay at its current node, provided that no two agents collide, i.e., vertex and swap conflicts are prohibited.

The objective is to complete all tasks as quickly as possible. A solution to MAPD is a set of conflict-free paths and task assignments, with each agent starting from $s_{i}$. When $|\Gamma|$ is finite, an MAPD algorithm is complete as it ensures that all tasks are finished within a finite number of timesteps.

Two kinds of objective functions are considered: (1) service time and (2) makespan, that is, the timestep when all tasks are completed. Note that makespan is only defined when $\Gamma$ is finite.

3.3 Other Notations

The diameter of graph $G$ is denoted by $\mathit{diam}(G)$, and its maximum degree by $\Delta(G)$. Let $\mathit{dist}\left(u,v\right)$ denote the shortest path length from $u\in V$ to $v\in V$.

4 Priority Inheritance with Backtracking (PIBT)

This section introduces the PIBT algorithm. The concept of PIBT is initially explained by intuitive examples. To avoid unnecessarily complex explanations, PIBT is introduced as a centralized algorithm, focusing on analyzing the prioritization scheme itself. Note that PIBT relies on a decoupled approach, rendering it readily amenable to decentralization, as briefly discussed later. After providing a theoretical analysis on PIBT including reachability, how to apply PIBT to specific problems, such as one-shot MAPF and MAPD, is described.

4.1 Concept

Aiming to solve MAPF iteratively, PIBT repeats one-timestep prioritized planning until it is terminated. At every timestep, each agent first updates its unique priority. Then, the agents sequentially determine their next location in decreasing order of priorities while avoiding nodes that have been requested by higher-priority agents. Prioritization alone, however, can still cause deadlocks. As shown in Fig. 1, a stuck agent $a_{1}$ cannot go anywhere without collisions with other agents; hence, this situation can be regarded as a certain kind of deadlock.

4.1.1 Priority Inheritance

A situation with a stuck agent can also be regarded as a case of priority inversion, in the sense that a low-priority agent ($a_{1}$) holding a resource claimed by a higher-priority agent ($a_{3}$) fails to obtain a second resource held by an agent with medium priority $a_{2}$. A classical way to deal with priority inversion is to rely on priority inheritance [20] (Fig. 1). The rationale is that a low-priority agent ($a_{1}$) temporarily inherits the higher priority of the agent ($a_{3}$) claiming the resources it holds, thus forcing medium-priority agents ($a_{2}$) out of the way (Fig. 1).

4.1.2 Backtracking

Priority inheritance deals effectively with priority inversion; however, it does not completely ensure deadlock freedom. For instance, as shown in Fig. 2, an agent $a_{3}$ cannot escape as a result of consecutive priority inheritances ($a_{7}\rightarrow a_{6}\rightarrow a_{5}\rightarrow a_{4}$).

The solution relies on backtracking; any agent $a_{i}$ that executes priority inheritance must wait for an outcome, valid or invalid. If valid, $a_{i}$ successfully moves to the desired node. Otherwise, it must request a different node, excluding: (1) nodes requested by a higher priority agent, and (2) nodes having already returned an invalid outcome. Upon finding no valid or unoccupied nodes, $a_{i}$ sends back an invalid outcome to the agent from which it inherited its priority.

For instance, in Fig. 2, $a_{3}$ sends “invalid” back to $a_{4}$ as the outcome of the request received previously from $a_{4}$ (Fig. 2). $a_{4}$ tries to replan its next location; however, in the absence of other alternatives, in turn sends invalid to $a_{5}$. Because $a_{1}$ has lower priority, $a_{5}$ can let $a_{1}$ inherit its priority as an alternative, which leads to a valid outcome (Fig. 2), and agents can move (Fig. 2).

By combining priority inheritance and backtracking, it is ensured that all agents are assigned to the next locations without collisions.

4.2 Algorithm

We formalize the concept in Algorithm 1, which consists of (1) a top-level procedure [Lines 7–11] calling (2) a recursive procedure [Lines 12–26] that executes priority inheritance and backtracking.

Top-level Procedure

PIBT first updates the priority $p_{i}\in\mathbb{R}_{\geq 0}$ for each agent $a_{i}$ [Line 7]. The rule is simple; increment $p_{i}$ when $a_{i}$ has not reached its goal $g_{i}$; otherwise, reset $p_{i}$ to $\epsilon_{i}\in[0,1)$. The tie-breaker $\epsilon_{i}$, which is distinct for agents, has the role of keeping $p_{i}$ unique among agents. Subsequently, according to priorities, the algorithm assigns locations to each agent $a_{i}$ without a next location $\pi_{i}[t+1]$ via the procedure $\mathsf{PIBT}$ [Line 10]. The termination of the algorithm is explained later.

Recursive Procedure

The procedure $\mathsf{PIBT}$ implements the concept of priority inheritance and backtracking. It has two arguments: the agent $a_{i}$ making a decision and an agent $a_{j}$ from which $a_{i}$ inherits its priority, or $\bot$ if there is none. The procedure returns invalid if $a_{i}$ becomes a stuck agent like the red agent $a_{3}$ in Fig. 2, otherwise returns valid. Once this procedure is called, $a_{i}$ must determine $\pi_{i}[t+1]$ from the ordered set of candidate nodes $C$. Here, $C$ consists of $\pi_{i}[t]$ and its neighbors [Line 13], sorted in increasing order of distance from the goal $g_{i}$ [Line 14]. To avoid unnecessary priority inheritance, the presence (or absence) of an agent is used as a tie-breaking rule. $C$ is filtered, excluding two kinds of nodes: (1) nodes already requested by someone (to avoid vertex conflict) [Line 16] and (2) $\pi_{j}[t]$ if $a_{j}\neq\bot$ (to avoid swap conflict) [Line 17]. If no nodes remain in $C$, this is the case of a stuck agent; then, $a_{i}$ must stay at the current location and the procedure returns invalid [Lines 24–25].

For each node $v\in C$, the procedure checks whether $a_{i}$ can move to $v$ in the next timestep. After reserving $v$ for $a_{i}$ [Line 18], the priority inheritance occurs from $a_{i}$ to $a_{k}$ when another agent $a_{k}$, which has not yet determined the next location, occupies $v$ [Lines 19–21]. If $a_{k}$ returns invalid, the next location for $a_{i}$ is replanned [Line 20], or else, $a_{i}$ finishes the planning and returns valid, as well as that $v$ is unoccupied by others [Line 22].

A step-by-step example of Algorithm 1 is illustrated in Fig. 3.

4.3 Theoretical Analysis

4.3.1 Reachability

Here, it is proven that all agents eventually reach their respective destination in graphs with adequate properties (e.g., biconnected graphs). The proof relies on the following Lemma 1, which states that the agent with the highest priority at each timestep is always assigned to its desired node, assuming that it can always move along a simple cycle.

Lemma 1**.**

Let $a_{1}$ denote the agent with highest priority at timestep $t$ and $v^{\ast}$ the nearest node to $g_{1}$ among neighbors of $\pi_{1}[t]$. If a simple cycle $\bm{C}=\left(\pi_{1}[t],v^{\ast},\ldots\right)$ exists, Algorithm 1 assigns $v^{\ast}$ to $a_{1}$ as $\pi_{1}[t+1]$.

Proof.

Fig. 4 visualizes the following.

Assume that the top-level procedure calls $\mathsf{PIBT}(a_{1},\bot)$ at Line 10 for the agent with highest priority. At this phase, $\pi_{i}[t+1]$ is $\bot$ for any agent $a_{i}$, i.e., the next location has not been assigned yet. Consequently, $a_{1}$ selects $v^{\ast}$ as a target node $v$ in the first iteration of Lines 9–11. Then, it remains to prove that Line 20, priority inheritance, never returns invalid when another agent $a_{2}$ occupies $v^{\ast}$. In the other cases, $a_{1}$ evident gets $v^{\ast}$.

Next, assume that $\mathsf{PIBT}(a_{2},a_{1})$ is called from the original $\mathsf{PIBT}(a_{1},\bot)$. The existence of the cycle $\bm{C}=(\pi_{1}[t],v^{\ast})$ guarantees that $a_{2}$ has a non-empty set of candidate nodes $C_{2}$ for the next timestep without colliding with $a_{1}$. During the iterations of Lines 15–23, $\mathsf{PIBT}(a_{2},a_{1})$ returns valid when an unoccupied node $v\in C_{2}$ at timestep $t$ is selected. This forms the basis of the remaining proof by induction.

Following this, suppose that $\mathsf{PIBT}(a_{i},a_{i-1})$ is called before $\mathsf{PIBT}(a_{2},a_{1})$ returns any value. Similar to the case of $a_{2}$, $\mathsf{PIBT}(a_{i},a_{i-1})$ must return valid when one of the surrounding nodes of $\pi_{i}[t]$ is unoccupied at timestep $t$. In addition, as distinct from $a_{2}$, which must avoid swap conflict with $a_{1}$, $a_{i}$ can select $\pi_{1}[t]$ and return valid when $\pi_{1}[t]$ is neighbor to $\pi_{i}[t]$. As a result, $\mathsf{PIBT}(a_{i-1},a_{i-2})$, which calls $\mathsf{PIBT}(a_{i},a_{i-1})$ also returns valid and eventually, $\mathsf{PIBT}(a_{2},a_{1})$ returns valid.

Next, by contradiction, suppose that $\mathsf{PIBT}(a_{2},a_{1})$ returns invalid. This assumption means that, for all agents $a_{j}(\neq a_{1})$ neighboring $a_{2}$ at timestep $t$, $\mathsf{PIBT}(a_{j},\cdot)$ returns invalid. This is the same for $a_{j}$, and so forth. Meanwhile, the existence of $\bm{C}$ indicates that at least one agent $a_{k}\neq a_{2}$ such that $\pi_{k}[t]\in\bm{C}$ had initially at least one free neighbor node. Even though all nodes in $\bm{C}$ are occupied, the agent on the last node of $\bm{C}$ has a free neighbor node, i.e., $\pi_{1}[t]$. This is a contradiction; $\mathsf{PIBT}(a_{k},\cdot)$ returns valid; consequently, $\mathsf{PIBT}(a_{2},a_{1})$ returns valid. ∎

Theorem 1**.**

Given an MAPF instance such that $G$ has a simple cycle for all pairs of adjacent nodes, PIBT (Algorithm 1) generates a set of conflict-free paths $\{\pi_{1},\ldots,\pi_{n}\}$ such that for any agent $a_{i}\in A$, $\pi_{i}[0]=s_{i}$ and there is a timestep $t\leq\mathit{diam}(G)\cdot|A|$ such that $\pi_{i}[t]=g_{i}$.

Proof.

From Lemma 1, the agent with highest priority reaches its own goal within $\mathit{diam}(G)$ timesteps. Based on the update rule of the priority $p_{i}$ for an agent $a_{i}$, once some agent $a_{i}$ has reached its goal, $p_{i}$ is reset and is lower than the priority of all other agents who have not reached their goal yet. These agents see their priority increase by one. As long as such agents remain, exactly one of them must have the highest priority. In turn, this agent reaches its own goal after at most $\mathit{diam}(G)$ timesteps. This repeats until all agents have reached their goal at least once, which takes at most $\mathit{diam}(G)\cdot|A|$ timesteps in total. ∎

Hereafter, this property is referred to as reachability, that is, all agents eventually reach their goal. Reachability differs from completeness for one-shot MAPF; it is never ensured that all agents reach their goal simultaneously; hence, PIBT is not ensured to solve conventional MAPF, defined in Section 3.1. Alternatively, with the graph condition of Theorem 1, PIBT is complete for the MAPF variant where agents need not necessarily stay at their goals.

A typical example that satisfies the aforementioned graph condition is the biconnected undirected graph. The opposite is not true, however, and Theorem 1 is expressed more generally to consider directed graphs. For instance, a directed ring satisfies the condition even though it is not biconnected.

Without the graph condition in Theorem 1, several agents may remain in the same vertices permanently and may not reach their goals. Figure 5 shows such an example. In practice, PIBT needs a pre-defined maximum planning timestep to detect such failure cases. When PIBT reaches this timestep, it stops planning and reports a planning failure. See also Section 4.4.1.

4.3.2 Complexity Analysis

Now we consider the time complexity of PIBT. Let $F$ be the maximum time required for sorting candidate nodes [Line 14]. $F$ depends on both $G$ and the distance computation for the goals.

Proposition 1**.**

The time complexity of PIBT in one timestep is $O\left(|A|\cdot(\Delta(G)+F+\lg{|A|})\right)$.

Proof.

Within one timestep, the procedure $\mathsf{PIBT}$ is called exactly $|A|$ times, once for each agent. This is because $\mathsf{PIBT}(a_{i},\cdot)$ is called if and only if an agent $a_{i}$ has not yet determined the next location (that is, when $\pi_{i}[t+1]=\bot$) [Lines 10, 20] but $\mathsf{PIBT}(a_{i},\cdot)$ assigns $\pi_{i}[t+1]$ [Line 18] before calling another $\mathsf{PIBT}(a_{k},a_{i})$ for priority inheritance. The loop of Line 15–23 iterates at most $\Delta(G)+1$ times. Each operation in the loop is performed in constant time. From the assumption, Line 14 requires $O(F)$. As a result, the procedure $\mathsf{PIBT}$ requires $O(|A|\cdot(\Delta(G)+F))$.

Line 7 requires $O(|A|)$. Line 8 requires $O(|A|\lg{|A|})$. In total, PIBT in one timestep requires $O\left(|A|\cdot(\Delta(G)+F+\lg{|A|})\right)$. ∎

The analysis is further continued for specific conditions. Assume that a distance table for the goal of each agent is computed by breadth-first search prior to executing PIBT, where the overhead is $O(|A|\cdot|E|)$. Then, $F$ will be solely sorting candidate nodes following the table, resulting in $O(\Delta(G)\lg{\Delta(G)})$. Consequently, PIBT in one timestep is $O\left(|A|\cdot(\Delta(G)\lg{\Delta(G)}+\lg{|A|})\right)$. Assume further that $G$ is a 4-connected grid, as commonly used in MAPF studies. Here, instead of $\Delta(G)$, consider $\Delta(G)+1$ for accurate analysis. Then, $(\Delta(G)+1)\lg{(\Delta(G)+1)}\simeq 11.6$. As a result, without a huge team of agents ($|A|<3125$), the first term $|A|\cdot\Delta(G)\lg{\Delta(G)}$ is dominant; otherwise, the second term $|A|\lg{|A|}$ is dominant. In either case, PIBT can be said to be computationally very inexpensive.

The small time complexity provides several advantages. For instance, PIBT can address large instances for both $A$ and $G$, which other MAPF algorithms cannot solve within a realistic timeframe. Another advantage is an application to anytime planning, a scheme that yields a feasible solution whenever interrupted but gradually improves solution quality as time goes by. Anytime planning is attractive, particularly in on-demand situations where the deliberation time is finite. A previous study realizes anytime planning for MAPF by refining known solutions iteratively [13]. For such an approach, obtaining initial solutions as quickly as possible is key because we can leave much time for refinement. In this sense, PIBT is a good choice to obtain an initial solution. In contrast, the anytime approach cannot be realized by “slow” solvers because they may not provide solutions within a time limit. Lastly, given a certain timestep, the runtime of PIBT is predictable, which is a fundamental characteristic in real-time planning. The power of PIBT is reflected in the experiment.

4.4 Application to Specific Problems

We now present how to adapt PIBT to solve typical pathfinding problems of multiple agents, namely, MAPF and MAPD.

4.4.1 Application to MAPF

For convenience, configuration refers to a set of locations of all agents at a given timestep. For instance, the initial configuration is $\left(s_{1},\ldots,s_{n}\right)$.

Termination

So far, when the PIBT run has been completed, has not been specified. In the conventional MAPF, PIBT is assumed to run until it reaches the goal configuration $(g_{1},\ldots,g_{n})$; otherwise, PIBT “fails” to solve the MAPF instance when it reaches the pre-defined timestep.

Distance evaluation

In PIBT, for each timestep, each agent has to evaluate distances from the surrounding nodes to its goal [Line 14 of Algorithm 1]. This operation could be implemented by calling $\text{A}^{\ast}$ on demand but it could also be a bottleneck. Instead, PIBT can save computation time by preparing distance tables from each agent’s goal. This is computed by breadth-first search from the goals with an overhead of $O(|A|\cdot|E|)$. Indeed, this is the main reason for the speedup of the implementation from our previous version of PIBT [21].

Extension for one-shot MAPF

Even if $G$ satisfies the condition for reachability, PIBT does not ensure that all agents reach their goal simultaneously, which is a requirement of conventional MAPF. In fact, with naive PIBT, a certain kind of livelock situation was confirmed in our experiment. This motivates the development of PIBT+, a framework that enhances known MAPF solvers, presented in Algorithm 2. The main idea behind PIBT+ is to make problem instances easier for the MAPF solver by bringing agents near their destinations using PIBT. In this sense, we say that the target MAPF solver is a complement solver.

The concept of Algorithm 2 is as follows. $T_{\text{min}}$ is the minimum number of timesteps needed for solutions from the initial configuration to the goal configuration. Because PIBT can be regarded as prioritized planning, the agent with the longest initial distance reaches its goal following the shortest path by providing priorities according to the initial distance. This implies that there is a chance that all agents reach their goals at $T_{\text{min}}$. The remaining problem is relatively easy compared to the original because most agents are expected to have already reached their goals.

Theorem 2**.**

PIBT+ is complete for MAPF on undirected graphs as long as the completeness conditions of the complement solver are satisfied.

Proof.

Let $C_{s},C_{t},C_{g}$ be the initial configuration, the configuration at $T_{\text{min}}$ planned by Line 5 in Algorithm 2, and the goal configuration, respectively. There exists a solution from $C_{t}$ to $C_{s}$ because PIBT computes the plan from $C_{s}$ to $C_{t}$. If the original problem is solvable, there exists a solution from $C_{s}$ to $C_{g}$, meaning that, at least one solution exists from $C_{t}$ to $C_{g}$. According to this assumption, PIBT+ finally uses the complete solver from $C_{t}$ to $C_{g}$, and the complete solver must return a solution. The above satisfies the statement. ∎

PIBT+ is essentially helpful for situations where it is difficult to obtain solutions through a direct adaptation of the complement solver but where plausible solutions are sought in a short time. As a complement solver, it is desirable to use complete algorithms such as rule-based, search-based, or compiling-based approaches. We note that PIBT+ does not guarantee a better solution nor finish planning faster than using the complement solver from the beginning. On the other hand, since the transition from the initial configuration to the configuration at $T_{\text{min}}$ is reversible, the additional burdens of sum-of-cost and makespan from optimal ones can be, respectively, at most $2|A|\cdot T_{\text{min}}$ and $2T_{\text{min}}$. Similarly, the additional burden of time complexity can be $O(T_{\text{min}}\cdot|A|\cdot(\Delta(G)+F+\log{|A|})$ from the consequence of Proposition 1.

4.4.2 Applying to MAPD

The MAPD problem is a typical example of online and lifelong scenarios requiring task allocation. We here describe how PIBT adapts to MAPD.

To design a complete MAPD algorithm, we must guarantee two claims: (1) all non-assigned tasks are eventually assigned, and (2) all assigned tasks are eventually completed, i.e., a task-assigned agent must visit pickup and delivery locations in order. For simplification, consider assigning a task to an agent only when the agent is at the pickup location. The remaining work for the agent is to visit the delivery location.

PIBT (Algorithm 1) is convenient for the second claim due to the following reason. Recall that Lemma 1 states that, for each timestep, the agent with the highest priority can always move to an arbitrary neighbor node. Using this lemma, ensuring that a task-assigned agent completes the task in finite time is straightforward. This is achieved with a special prioritization scheme that satisfies the two conditions: (1) all task-assigned agents have higher priorities than free agents, and (2) every task-assigned agent eventually gets the highest priority and holds it until the task is completed.

The first claim, all non-assigned tasks are eventually assigned, is achieved by various approaches. We here take a simple approach, i.e., for each timestep, all free agents set their goals to the nearest pickup location of non-assigned tasks. Assume that every free agent eventually gets the highest priority among free agents and holds the highest until it reaches a goal (a pickup location). Then, this approach satisfies the first claim because all task-assigned agents eventually become free due to the second claim.

Algorithm 3 realizes the aforementioned concept to solve MAPD. For each timestep, the algorithm first assigns tasks while updating the agents’ goals [Lines 4–14]. Then, the algorithm uses PIBT to plan locations for the next timestep [Lines 15–16]. As discussed earlier, for each timestep, Algorithm 3 prioritizes the task-assigned agents than free agents. As a secondary prioritization, it uses the original PIBT prioritization scheme (Line 7 of Algorithm 1) as it is.

The following theorem is a consequence of the aforementioned discussion and its reachability.

Theorem 3**.**

Algorithm 3 is complete for an MAPD problem when $G$ has a simple cycle for all pairs of adjacent nodes.

Comparison of our Proposed Approach with an Existing Complete Approach

We later compare Algorithm 3 with the TP algorithm [18] experimentally. Before that, we here provide a qualitative discussion as follows. TP is complete for MAPD in a limited situation. First, it requires locations that never be either pickup or delivery locations for any tasks, called non-task endpoints. The non-task endpoints are required at least for the number of agents. Then, for any two endpoints, a path must exist without traversing any other endpoints. Here, endpoints consist of non-task endpoints and candidates of pickup/delivery locations. Compared to TP, PIBT does not require such conditions and works in a wide range of situations as long as the graph condition is satisfied.

Other Task Allocation

We took a simple and greedy task allocation process; however, more aggressive approaches are applicable, e.g., incorporating linear cost-optimal assignment [63] or bottleneck assignment [64]. Doing so can improve solution qualities but potentially requires more computation time.

4.5 Decentralized Online Planning

Here, the decentralization of PIBT implementations is briefly discussed. In particular, we focus on the online planning of PIBT, that is, agents repeat a set of single-timestep planning and move actions. Because PIBT is based on prioritized planning, adopting a decentralized context is realistic. The part of priority inheritance and backtracking is performed by the propagation of information. Further, PIBT relies on local interactions between agents, implying that agents are not required to know other agents’ information far away from their current location. To illustrate this, the concept of interacting agents, a set of agents that must negotiate their planning, is introduced.

Interacting Agents

When two agents are located within two hops of each other’s move, they may collide in the next timestep; then, they are said to be directly interacting in that timestep. For an agent $a_{i}$, a group of interacting agents $\mathcal{A}_{i}(t)\subseteq A$ is then defined by transitivity over direct interactions in timestep $t$. Note that, given $a_{i}$ and $\mathcal{A}_{i}(t)$, for any other agent $a_{j}\in\mathcal{A}_{i}(t)$, we have $\mathcal{A}_{j}(t)=\mathcal{A}_{i}(t)$. Whenever the context is obvious, $\mathcal{A}$ is directly used.

Because agents belonging to different groups cannot affect each other, path planning, the negotiation process using priority inheritance and backtracking can effectively occur in parallel. As a result, theoretically speaking, PIBT can be decentralized, relying only on local interactions, that is, it is sufficient that two agents in close proximity talk directly and utilize multi-hop communication; however, note that the groups of interacting agents are fully identified prior to starting a PIBT process for one timestep.

Communication

Assume that interacting agents are fully detected at each timestep. Then, we analyze communication complexity, i.e., how many messages are required in PIBT. In a decentralized context, PIBT requires agents to know others’ priorities before deciding on the next nodes. Usually, this requires $|\mathcal{A}|^{2}$ communication between agents; however, the update rule of the priority $p_{i}$ relaxes this effort, for instance, storing other agents’ priorities and communicating only when the priority is reset. Therefore, the communication cost of PIBT mainly depends on the information propagation phase.

Next, consider this information propagation phase. In PIBT, communication between agents corresponds to calling $\mathsf{PIBT}$ (priority inheritance), and when the procedure returns a value (corresponding to backtracking). $\mathsf{PIBT}(a_{i},\cdot)$ is never called twice within a timestep, as discussed in the analysis of time complexity. Moreover, each agent sends a backtracking message at most once in each timestep. Overall, the communication cost for PIBT at each timestep is linear for the number of agents, that is, $O(|A|)$. In reality, this can be even lower because the figures depend on the number of interacting agents $|\mathcal{A}|$, which can be much smaller than $|A|$.

4.6 Without Rotations

In practice, in physical environments such as mobile robots, movements corresponding to “rotations” might be difficult to realize due to synchronization problems. A rotation is a set of adjacent agents moving along a circle within one timestep. Formally, a rotation occurs for a subset of agents $\{a_{i},a_{j},a_{k},\ldots,a_{l}\}\subseteq A$ during timestep $t$ and $t+1$ when $\pi_{i}[t+1]=\pi_{j}[t]\land\pi_{j}[t+1]=\pi_{k}[t]\land\ldots\pi_{l}[t+1]=\pi_{i}[t]$ is satisfied. We refer to a set of paths as rotation-free when there is no rotation in the paths. This part adapts PIBT to situations where rotations are prohibited.

PIBT does not require major changes even in a model without rotations. During the iteration for the candidate nodes of the next timestep [Lines 15–23], skip nodes resulting in rotations like Lines 16 or 17 to prevent collisions. We denote the corresponding Algorithm 1 as PIBT*⨂*. This variant outputs a set of rotation-free paths.

Lemma 2**.**

Let $a_{1}$ denote the agent with highest priority at timestep $t$ and $v^{\ast}$ the nearest node to $g_{1}$ neighboring $\pi_{1}[t]$. If $|A|<|V|$ and there exists a path between $v^{\ast}$ to an arbitrary node without going through $\pi_{1}[t]$, PIBT⨂ assigns $v^{\ast}$ to $a_{1}$ as $\pi_{i}[t+1]$.

Proof.

Following $|A|<|V|$, we can derive $\exists v^{\prime}\not\in\{\pi_{i}[t]\,|\,a_{i}\in A\}$. There exists a path $\bm{D}$ between $v^{\ast}$ to $v^{\prime}$ without going through $\pi_{1}[t]$. The subsequent proof is almost identical to that of Lemma 1 and uses $\bm{D}$ instead of $\bm{C}$. ∎

Theorem 4**.**

Given an MAPF instance such that $G$ is biconnected and $|A|<|V|$, PIBT⨂ generates a set of conflict-free and rotation-free paths $\{\pi_{1},\ldots,\pi_{n}\}$ such that, for any agent $a_{i}\in A$, $\pi_{i}[0]=s_{i}$ and there is a timestep $t\leq\mathit{diam}(G)\cdot|A|$ such that $\pi_{i}[t]=g_{i}$.

Proof.

The same proof procedure of the reachability (Theorem 1) can be applied using Lemma 2 instead of Lemma 1. ∎

Note that this theorem is restricted to biconnected graphs compared to the original theorem on reachability.

5 Evaluation

This section evaluates PIBT thoroughly to empirically demonstrate that PIBT is a quick and scalable MAPF algorithm with acceptable solution quality. PIBT is tested in MAPF (Section 5.1) and MAPD (Section 5.2), followed by a stress test for the number of agents (Section 5.3) and robot demonstration (Section 5.4). The simulator was developed in C**++**, and the experiments were run on a laptop with Intel Core i9 2.3GHz CPU and 16GB RAM. The code is available at https://kei18.github.io/pibt2. We also provide the experimental result of PIBT for MAPF in extremely dense situations in Section B of the Appendix.

5.1 Multi-Agent Path Finding (MAPF)

5.1.1 Setup

Benchmark

The MAPF benchmark [1], which includes a set of four-connected grids and start–goal pairs for agents, was used. Ten grids were first selected with different portfolios, e.g., size, sparseness, and complexity (see Fig. 11, 12 in the Appendix). For each grid, 25 “random scenarios” were used while increasing the number of agents by ten up to the maximum (1000 agents in most cases). Therefore, identical instances were tried for the solvers in all settings. Note that almost all selected grids did not satisfy the graph condition of reachability. The problem setting follows conventional MAPF, i.e., agents are eventually on their goals simultaneously.

Baselines

The following algorithms were carefully picked for comparisons with PIBT in MAPF:

• •

HC$\text{A}^{\ast}$ [4] as standard prioritized planning [19]. Aiming at improving solution quality, we prioritized each agent so that those with more considerable distances from their starts to goals have higher priorities, following heuristics from [53].

• •

EECBS [15] as a state-of-the-art bounded search-based sub-optimal solver; the sub-optimality was set to $1.2$. Note that EECBS is bounded sub-optimal for the sum-of-costs metric.

• •

EECBS-M: The adapted version of EECBS for the makespan metric.

• •

Push and Swap (PS) [14] as a standard rule-based approach: although PS is incomplete as pointed out in [40], it is the origin of many extended algorithms [56, 40, 57, 59]; hence, PS was used.

• •

PS+: The post-processing solutions were obtained by PS. PS allows at most one agent to move, resulting in terrible outcomes. Thus, the solutions were compressed while preserving temporal dependencies of the solutions, influenced by the techniques in [65], which allow multiple agents to move simultaneously.

• •

CBS [16] with improvement techniques [66, 67, 68, 69, 70, 71, 12] as a state-of-the-art search-based optimal solver. Note that CBS optimizes the sum-of-costs metric.

• •

CBS-M: The makespan optimal version of CBS instead of sum-of-costs.

• •

BCP [17, 11] as a state-of-the-art compiling-based optimal solver, which is optimal for the sum-of-costs metric.

For EECBS, CBS, and BCP, the implementations coded by their respective authors were used.222 The codes are available on https://github.com/Jiaoyang-Li/EECBS, https://github.com/Jiaoyang-Li/CBSH2-RTC, and https://github.com/ed-lam/bcp-mapf, respectively. (EE)CBS-M was implemented in a best-first search manner that prioritizes makespan-better search nodes in the high-level tree, modifying the aforementioned codes.

When these implementations have parameter specifications, the default was used for each. The other solvers (HC$\text{A}^{\ast}$, PS, and PS+) were coded by the author, which is own-coded in C**++**.

Failure Case

An algorithm was regarded as having failed to solve an instance when either of the three conditions was met: (1) The algorithm reported failure. (2) The algorithm reached the runtime limit (30 seconds). This value was based on [1]. (3) The algorithm reached the makespan limit (2000 in brc202d; otherwise 1000). This rules out impractical MAPF outcomes. The values were set according to preliminary results.

Metrics

The following four evaluation metrics were used, referring to other MAPF studies: (1) success rate over 25 instances, (2) runtime, (3) sum-of-costs divided by $\sum_{a_{i}\in A}\mathit{dist}\left(s_{i},g_{i}\right)$, and (4) makespan divided by $\max_{a_{i}\in A}\mathit{dist}\left(s_{i},g_{i}\right)$. The latter two assess the solution quality; smaller solutions are better, and a minimum of one solution is necessary. Although optimal costs are challenging to obtain, these scores work as a lower bound of sub-optimality. Note that CBS and BCP are optimal for the sum-of-costs metric whereas CBS-M is optimal for makespan; we hence omit to plot makespan scores of CBS/BCP and sum-of-costs scores of CBS-M. Sum-of-costs and makespan have Pareto optimal structure [7]; hence, generally, they cannot be minimized simultaneously.

Other Remarks

PIBT+ uses PS+ as a complement solver. Own-coded solvers (PIBT*(+), HC$\text{A}^{\ast}$, PS(+)*) used a distance table for the start locations of each agent, computed by breadth-first search. The runtime included the procedure of creating the table.

5.1.2 Result

Figure 6 presents part of the results. The full version is displayed in Fig. 11 and 12 in the Appendix. Excluding success rates, the scores are averaged over only the successful instances with that solver.333 For a fair comparison, it is better to present the average scores of instances that all solvers solved. However, due to the high failure rates, the current style of data plots was selected.

In addition, we show the classification of failures of each solver in Table 1.

Overall, PIBT outputs solutions immediately ($\leq$ 5 seconds in large maps) even with a thousand agents, with a slight decline in the solution quality (sum-of-costs). Specific findings are summarized as follows:

PIBT is extremely scalable

The scalability here is defined by map size and the number of agents. In both aspects, PIBT outperforms the other baselines. The runtime of PIBT is significantly faster by orders of magnitude compared to the other solvers. In other words, PIBT can solve large instances that other solvers cannot solve. This is owing to the small-time complexity.

PIBT outputs solutions with acceptable quality in sparse situations

In general, PIBT does not guarantee solution quality. However, both sum-of-costs and makespan are adequate compared to others in sparse settings. This is in contrast to pure rule-based solvers (PS*(+)*) that mostly fail due to the makespan limit even with fewer agents. PIBT is based on prioritized planning, enabling output solutions with acceptable quality.

Failure reasons of PIBT

Table 1 reveals that the failure reason for PIBT is due to the makespan limit. We further explain two failure categories of PIBT as follows. The first case is due to the nature of reachability (Theorem 1) that does not ensure that all agents reach their goals simultaneously. The second case is due to graphs without a cycle for any two adjacent nodes. In such graphs, it is possible to reach situations where several agents stop moving, similar to Fig. 5. As those agents remain in their locations, PIBT eventually reaches the makespan limit. Therefore, PIBT often fails with many agents in small maps (for instance, empty-48-48 and random-64-64-20).

PIBT+ can increase the success rate of PIBT dramatically

The aforementioned shortcoming of PIBT for one-shot MAPF is overcome by adding a complement solver. Note that PIBT+ sometimes failed because an incomplete solver PS+ was used as a complement solver. Note further that it is possible to use other solvers for the complement solver.

Solution quality of PIBT+ in dense situations

In Fig. 6, the upper bound of suboptimality of PIBT+ in random-32-32-20 dramatically increases with the number of agents. This is due to the complement solver. In the experiment, PIBT+ used PS+ as the complement solver to achieve high success rates within a small computation time. In general, PS+ is quick even for large instances, however, its solution quality is not excellent, as seen in our results. In cluttered and dense situations such as random-32-32-20, it is difficult to obtain solutions by PIBT. Then, PIBT+ often uses the complement solver, degrading the solution quality.

Interpretations of comparisons’ results

Optimal solvers (CBS and BCP) can address a few hundred agents in some cases, but usually, they cannot address several hundreds of agents, emphasizing the necessity of sub-optimal solvers. Pure rule-based solvers (PS*(+)) are quick, but their solution qualities are not acceptable compared to the other solvers. Indeed, failures of PS(+)* are mainly due to the makespan limit. Prioritized planning (HC$\text{A}^{\ast}$) outputs plausible solutions for both sum-of-costs and makespan. It is scalable but not so much as PIBT. In addition, when scenarios are dense, it is significantly difficult to find solutions according to static priorities (see random-32-32-20 in Table 1), making dynamic priorities attractive as used in PIBT. The overall result of the search-based sub-optimal solver (EECBS) is similar to HC$\text{A}^{\ast}$, i.e., it outputs plausible solutions but is not as quick and scalable as PIBT. In short, PIBT has a unique position compared to other established MAPF approaches, i.e., it is rapid and scalable with acceptable solution qualities.

5.2 Multi-Agent Pickup and Delivery (MAPD)

5.2.1 Setup

The experimental setup follows that of the original MAPD study [18], using the same undirected graph as a testbed, and setting the same locations as candidates for pickup and delivery (Fig. 7). A sequence of 500 tasks was generated by randomly choosing pickup and delivery locations from all task endpoints. The six different task frequencies where numbers of tasks are added to the task set $\Gamma$ were as follows: 0.2 (one task every five timesteps), 0.5, 1, 2, 5, and 10 with the number of agents increasing from 10 to 50. The Token Passing (TP) algorithm [18] was also tested as a baseline, which we programmed in C**++**. All experimental settings were performed over 100 instances in which initial positions of agents were set randomly. The following three metrics were evaluated: (1) runtime per one timestep; MAPD is assumed to be online, (2) service time: from issue to completion of tasks, and (3) makespan: the first timestep at which all tasks are completed. Both PIBT (Algorithm 3) and TP used an all-pairs distance matrix, pre-computed with the Floyd–Warshal algorithm [72].

5.2.2 Result

Figure 8 summarizes the results. PIBT significantly outperforms TP in runtime and is comparable to or better than TP in solution quality, characterized by service time and makespan. The runtime improvements are because of the low time complexity of PIBT. The improvements in solution quality are mainly due to how the algorithm deals with free agents. Assume that there is one unassigned task. In PIBT, all free agents move toward the pickup location, with only the earliest one actually getting the task and then having to start moving to the delivery location while pushing the other free agents away, thanks to the prioritization scheme. In contrast, TP evacuates all free agents to non-task endpoints to avoid deadlocks other than the one agent that is newly assigned. However, this assigned agent must still “dodge” those free agents’ locations to prevent deadlocks. As a result, the path planned by PIBT for the task-assigned agent will be shorter than that of TP.

5.3 Stress Test for Scalability

5.3.1 Setup

The scalability of PIBT was evaluated through MAPF with a large map while varying the number of agents from $2,000$ to $10,000$. The map, shown in Fig. 9, was the largest in the MAPF benchmark [1]. Here, only runtime was evaluated per timestep, averaged over the first $100$ timesteps, regardless of whether the planning succeeded. Similar to the previous MAPF experiment, distance tables computed before executing PIBT were used. The starts and goals of agents were set randomly for each repetition.

5.3.2 Result

Figure 9 summarizes the result. The scores follow almost a linear trend in the number of agents. Surprisingly, even with $10,000$ agents, PIBT scored around ten milliseconds per one timestep on an ordinary laptop.

5.4 Demonstrations with Real Robots

PIBT was also implemented with a team of small physical robots. Here, an online and lifelong scenario is presented where a new goal is immediately assigned to an agent who reaches its current goal. Note that, because of reachability, PIBT ensures that all assigned goals are eventually met, that is, visited by assigned robots.

Platform

Toio robots (https://toio.io/) were used to implement PIBT. The robots, connected to a master computer via the Bluetooth LE (low energy) protocol, advance on a specific playmat and are controllable by instructions using absolute coordinates.

Usage

The robots were controlled in a centralized, synchronized, and online mode, described as follows. A virtual grid ($7\times 5$ with obstacles) was created on the playmat; the robots followed the grid. A central server (a laptop) managed the locations of all robots. Periodically, the server executed PIBT for one single timestep and issued the instructions (that is, where to go) to each robot. The code was written in Node.js.

Snapshot

Figure 10 shows a snapshot of the demo. The full video and code are also available at https://kei18.github.io/pibt2.

6 Conclusion

This study introduces PIBT, a scalable algorithm for solving MAPF iteratively. PIBT focuses on the adjacent movements of multiple agents, relying on a simple prioritization scheme; hence, it can be applied to several domains, including online and lifelong situations. The four empirical results support this aspect: (1) PIBT*(+)* can promptly solve large MAPF instances, whereas other solvers take the time or require unrealistic computational times. (2) PIBT outperforms current solutions for pickup and delivery (MAPD). (3) Despite thousands of agents, PIBT can yield planning in a real-time manner. (4) A real-robot execution of PIBT was further presented. We believe that PIBT provides practical insights into controlling a swarm of agents in practical situations.

Future directions for research in this field include jumping into the “wild” environment, that is, considering asynchrony. In traditional MAPF settings, movement between agents is assumed to be synchronized. Although this assumption enables the agents’ behavior control, there exists a significant gap in practical situations mainly because of the asynchrony of the agents’ moves, which requires further consideration. Our recent study [73] presents other directions.

Acknowledgments

We thank the anonymous reviewers for their many insightful comments. This research is partly supported by JSPS KAKENHI Grant Numbers 20J23011, 21K11748, and 21H03423. Keisuke Okumura would like to thank the Yoshida Scholarship Foundation for their support. We thank Editage (www.editage.com) for English language editing.

Appendix

Appendix A MAPF Results

Figures 11 and 12 summarize all results of the MAPF experiments in Section 5.1.

Appendix B PIBT in Extremely Dense Situations

This section presents additional experiments on PIBT in extremely dense situations where $|A|>|V|/2$ and $G$ satisfies the graph condition of Theorem 1. We used empty-8-8 and the same experimental settings as those introduced in Section 5.1.

Table 2 summarizes the result. Most instances are solved at most 5 ms; otherwise, PIBT failed by reaching the makespan limit. Solution qualities dramatically increase with more agents because, in dense situations, most agents cannot take their shortest paths.

Interestingly, PIBT solved all instances with $64$ agents (fully occupied), while sometimes failing with $50$ agents. This is because of the randomness of the tie-break rule at Line 14 of Algorithm 1. As a discussion of the implementation level, we used three rules when sorting candidate nodes for the following location: (1) distances to the goal, (2) the presence of agents to avoid unnecessary priority inheritance, and (3) random values, when neither the prior two break a tie. We also tested the rule-2 omitted version, succeeding for all instances regardless of $|A|$. Since the rule-2 loses its meaning in situations where $|A|=|V|$, we consider that the randomness contributes to solving extremely dense situations. We also observed that this trend is the same when testing a $16\times 16$ empty grid, i.e., usual PIBT sometimes failed whereas PIBT that omits the rule-2 tie-break solved all instances regardless of $|A|$. From these observations, we guess that in tidy environments like empty-8-8, PIBT without the rule-2 can solve one-shot MAPF with a high probability as makespan increases. This is an interesting direction to seek but significantly beyond this paper.

Note that the aforementioned discussion is not applicable when $G$ does not satisfy the graph condition of Theorem 1. Moreover, rule-2 is usually effective in improving sum-of-costs, as shown in Table 3.