CARE: Cooperative Autonomy for Resilience and Efficiency of Robot Teams for Complete Coverage of Unknown Environments under Robot Failures

TL;DR

This paper introduces CARE, a distributed multi-robot coverage algorithm that enhances resilience to robot failures and improves efficiency through event-driven replanning and game-theoretic decision-making.

Contribution

The paper presents a novel distributed algorithm using potential games for resilient and efficient multi-robot coverage in unknown environments with robot failures.

Findings

Achieves complete coverage despite robot failures

Reduces coverage time compared to alternative methods

Enables faster target discovery in complex scenarios

Abstract

This paper addresses the problem of Multi-robot Coverage Path Planning (MCPP) for unknown environments in the presence of robot failures. Unexpected robot failures can seriously degrade the performance of a robot team and in extreme cases jeopardize the overall operation. Therefore, this paper presents a distributed algorithm, called Cooperative Autonomy for Resilience and Efficiency (CARE), which not only provides resilience to the robot team against failures of individual robots, but also improves the overall efficiency of operation via event-driven replanning. The algorithm uses distributed Discrete Event Supervisors (DESs), which trigger games between a set of feasible players in the event of a robot failure or idling, to make collaborative decisions for task reallocations. The game-theoretic structure is built using Potential Games, where the utility of each player is aligned with a shared objective function for all players. The algorithm has been validated in various complex scenarios on a high-fidelity robotic simulator, and the results demonstrate that the team achieves complete coverage under failures, reduced coverage time, and faster target discovery as compared to three alternative methods.

Tables

Table 1. Table I: A comparison of key features with other online MCPP algorithms

	CARE	First-responder [9]	Brick & Mortar [18]	ORMSTC [19]
Path Pattern	Back and forth	Back and forth	No obvious pattern observed	Spiral
Resilience Strategy	Neighbors jointly optimize to reorganize themselves to immediately fill the coverage gap caused by the failed robot if the optimization criteria are satisfied	Wait until some robot finishes its task and is reassigned to fill the coverage gap	Remaining robots continue re- gularly. The coverage gaps be- come extra workloads. May pro- duce strongly overlapped paths due to the looping problem	Neighbors extend their trees to fill the coverage gap of the failed robot, but the approach is not proactive and the already explored area by the failed robot is scanned again
No-idling Strategy	The idling robot and its near-finishing neighbors jointly optimize to help other robots to reduce coverage time and collect more worth early	Idling robots are reallocated to new tasks that maximize their own utility	None	None
Optimization Factors	Estimated worths of contested tasks, remaining reliability and traveling time of live robots	Unexplored portion of tasks and traveling time of robots	None	None

Table 2. Table II: List of key parameters in CARE

Parameter	Description
$N$	Total number of robots
$M$	Total number of tasks
${\rho }_0$	Curvature of the growth part in battery model
${\rho }_1$	Inflection point in battery model
$u$	Robot traveling speed
$\omega$	Robot tasking speed
${\lambda }_r$	Expected number of targets in task $r$
$\eta$	Threshold to identify robots that are close to finishing their tasks
$\gamma$	Threshold to identify incomplete tasks with sufficient work left
${\kappa }_1$	Neighborhood size in no-idling games
${\kappa }_2$	Neighborhood size in resilient games

Table 3. Table III: Effects of varying team size

$N$	$CT$	$RR$
		min	mean	max
$4$	$2258.6s$	$0.071$	$0.086$	$0.109$
$6$	$1338.2s$	$0.546$	$0.610$	$0.672$
$8$	$984.71s$	$0.777$	$0.826$	$0.883$
$10$	$747.4s$	$0.876$	$0.904$	$0.926$

Equations

$$R_{v_{\ell }}(t)=\frac{1}{1+e^{{\rho }_0(t-{\rho }_1)}},$$

$${\mathcal{R}}^a\subseteq \bigcup _{\ell =1}^N\bigcup _{k=1}^K{ϵ}_{\ell }(k).$$

$$CR=\frac{\big{(}\cup_{\ell=1}^{N}\cup_{k=1}^{K}\epsilon_{\ell}(k)\big{)}\cap\mathcal{R}^{a}}{\mathcal{R}^{a}}\in[0,1].$$

$$H=(X,\mathcal{E},\delta ,x_0,X_m),$$

$${\mathcal{U}}_i(a_i^{\star },a_{-i}^{\star })=\underset{a_i\in {\mathcal{A}}_i}{\max }{\mathcal{U}}_i(a_i,a_{-i}^{\star }),\forall {\mathcal{P}}_i\in \mathcal{P}.$$

$${\mathcal{U}}_i(a_i^{\prime },a_{-i})-{\mathcal{U}}_i(a_i^{\prime \prime },a_{-i})=\varphi (a_i^{\prime },a_{-i})-\varphi (a_i^{\prime \prime },a_{-i}),$$

$$\phi(a_{\mathscr{P}})=\sum\limits_{r\in\tilde{\mathscr{A}}}w_{r}\bigg{(}1-\underset{\mathscr{P}_{i}\in\{\mathscr{P}\}_{r}}{\prod}\big{[}1-p_{r}(\mathscr{P}_{i})\big{]}\bigg{)},$$

$$p_r({\mathcal{P}}_i)=R_{{\mathcal{P}}_i}(\tilde{t}),$$

$$\tilde{t}=t_k+t_{tr}+t_r,$$

$$Pr\big{(}\mathbf{x_{r}}=x\big{)}=e^{-\lambda_{r}}\cdot\frac{{\lambda_{r}}^{x}}{x!},\ x=0,1,2\ldots$$

$$\widetilde{w_r}$$

$$=\begin{array}{rl}& \sum _{x=0}^{\infty }x\cdot e^{-{\lambda }_r}\cdot \frac{{{\lambda }_r}^x}{x!}-\xi \cdot \sum _{x=0}^{\infty }{e}^{-{\lambda }_r}\cdot \frac{{{\lambda }_r}^x}{x!}-\sum _{x=0}^{\xi }(x-\xi )\cdot e^{-{\lambda }_r}\cdot \frac{{{\lambda }_r}^x}{x!}\end{array}$$

$$\widetilde{w_r}=({\lambda }_r-\xi )+e^{-{\lambda }_r}\cdot \sum _{x=0}^{\xi }(\xi -x)\cdot \frac{{{\lambda }_r}^x}{x!}$$

$$w_{r}=\tilde{w_{r}}\cdot\big{(}1-q(r)\big{)}.$$

$${\mathcal{MC}}_i=\varphi (a_i,a_{-i})-\varphi (\varnothing ,a_{-i}),$$

$${\mathcal{U}}_i(a_i,a_{-i})$$

$$\displaystyle=\begin{aligned} &\sum\limits_{r\in\tilde{\mathscr{A}}}w_{r}\bigg{(}1-\underset{\mathscr{P}_{j}\in\{\mathscr{P}\}_{r}}{\prod}\big{[}1-p_{r}(\mathscr{P}_{j})\big{]}\bigg{)}\\ &-\sum\limits_{r\in\tilde{\mathscr{A}}}w_{r}\bigg{(}1-\underset{\mathscr{P}_{j}\in\{\mathscr{P}\}_{r}\setminus\mathscr{P}_{i}}{\prod}\big{[}1-p_{r}(\mathscr{P}_{j})\big{]}\bigg{)}\end{aligned}$$

$$=\begin{array}{rl}& \sum _{r\in \tilde{\mathcal{A}}}{w}_r\cdot \prod _{{\mathcal{P}}_j\in \{\mathcal{P}{\}}_r\setminus {\mathcal{P}}_i}[1-p_r({\mathcal{P}}_j)]\\ & -\sum _{r\in \tilde{\mathcal{A}}}{w}_r\cdot \prod _{{\mathcal{P}}_j\in \{\mathcal{P}{\}}_r}[1-p_r({\mathcal{P}}_j)]\end{array}$$

$${\mathcal{U}}_i(a_i,a_{-i})$$

$$=\begin{array}{rl}& w_{a_i}\cdot \prod _{{\mathcal{P}}_j\in \{\mathcal{P}{\}}_{a_i}\setminus {\mathcal{P}}_i}[1-p_{a_i}({\mathcal{P}}_j)]\\ & -w_{a_i}\cdot [1-p_{a_i}({\mathcal{P}}_i)]\prod _{{\mathcal{P}}_j\in \{\mathcal{P}{\}}_{a_i}\setminus {\mathcal{P}}_i}[1-p_{a_i}({\mathcal{P}}_j)]\end{array}$$

$$=\begin{array}{rl}& w_{a_i}\cdot p_{a_i}({\mathcal{P}}_i)\cdot \prod _{{\mathcal{P}}_j\in \{\mathcal{P}{\}}_{a_i}\setminus {\mathcal{P}}_i}[1-p_{a_i}({\mathcal{P}}_j)]\end{array}$$

$$\varphi (a_i^{\prime },a_{-i})-\varphi (a_i^{\prime \prime },a_{-i})$$

$$=(\varphi (a_i^{\prime },a_{-i})-\varphi (\varnothing ,a_{-i}))-(\varphi (a_i^{\prime \prime },a_{-i})-\varphi (\varnothing ,a_{-i}))$$

$$={\mathcal{U}}_i(a_i^{\prime },a_{-i})-{\mathcal{U}}_i(a_i^{\prime \prime },a_{-i})$$

$$a_i(k+1)=\{\begin{array}{ll}\widehat{a_i}(m),& \text{with probability }\mu \\ a_i(m),& \text{with probability }1-\mu .\end{array}$$

$$\Phi(a)=\sum\limits_{r=1}^{M}\tilde{w_{r}}\Big{(}1-\prod_{v_{\ell}\in\{V\}_{r}}[1-p_{r}(v_{\ell})]\Big{)},$$

$$\Phi (a)$$

$$\displaystyle=\begin{aligned} &\sum\limits_{r=1}^{M}\tilde{w_{r}}\Big{(}1-[1-p(r)]\cdot[1-q(r)]\Big{)}\end{aligned}$$

$$\Phi (a)$$

$$=\begin{array}{rl}& \sum _{r=1}^M\widetilde{w_r}\cdot [1-q(r)]\cdot p(r)+\sum _{r=1}^M\widetilde{w_r}\cdot q(r)\end{array}$$

$$=\begin{array}{rl}& \sum _{r=1}^M{w}_r\cdot p(r)+\sum _{r=1}^M\widetilde{w_r}\cdot q(r)\end{array}$$

$$\displaystyle=\begin{aligned} &\bigg{(}\sum_{r\in\tilde{\mathscr{A}}}w_{r}\cdot p(r)+\sum_{r\notin\tilde{\mathscr{A}}}w_{r}\cdot p(r)\bigg{)}+\sum\limits_{r=1}^{M}\tilde{w_{r}}\cdot q(r)\end{aligned}$$

$$=\begin{array}{rl}& \varphi (a_{\mathcal{P}})+\sum _{r\notin \tilde{\mathcal{A}}}{w}_r\cdot p(r)+\sum _{r=1}^M\widetilde{w_r}\cdot q(r)\end{array}$$

$$G_P=\frac{\varphi (a_{\mathcal{P}}^{\star })-\varphi (a_{\mathcal{P}})}{{\sum }_{r\in \tilde{\mathcal{A}}}{w}_r}\in [0,1].$$

$$G_T=\frac{\Phi (a^{\star })-\Phi (a)}{{\sum }_{r=1}^M\widetilde{w_r}}\in [0,1].$$

$$\text{centering}\{\begin{array}{ll}\frac{1}{1+e^{{\rho }_0(780-{\rho }_1)}}& =0.9\\ \frac{1}{1+e^{{\rho }_0(2\times 780-{\rho }_1)}}& =0.4\end{array},\text{@add@centering}$$

Full text

1 Introduction

The search and coverage operations of autonomous robots have widespread applications such as floor cleaning, lawn mowing, oil spill cleaning, crop cutting and seeding, mine countermeasures, ocean floor inspection. These operations require Coverage Path Planning (CPP) [1][2][3][4][5], where a coverage path is needed for the robot to completely cover the search area while avoiding obstacles and having minimum overlapping trajectory to minimize the total coverage time.

Thus far, several CPP algorithms have been reported in literature [6][7] and a brief review is provided in Section 2.

1-A Motivation

Although many CPP methods are available when using a single robot, only a limited body of work has focused on Multi-robot Coverage Path Planning (MCPP). A popular control architecture in the existing MCPP methods is to split the overall workload into multiple tasks, and then use some single-robot CPP method for coverage in each task [8][9].

However, since the robots typically operate in uncertain environments, they are prone to different failures such as sensor or actuator malfunctions, mechanical defects, loss of power [10]. The consequences of these failures include coverage gaps, loss of critical data, performance degradation (e.g., missed detections of targets), prolonged operation time, and in extreme cases overall mission failure. For example, coverage gaps in mine countermeasure operations can leave undetected underwater mines which are serious threats to traversing vessels. It is therefore critical that the robot team is resilient to failures, in the sense that it can sustain the overall team operation and protect the mission goals (e.g., complete coverage) even in presence of a few robot failures [11]. The role of resilience is to assure system-level survivability and fast recovery to normalcy from unanticipated emergency situations (e.g., robot failures). In the context of the MCPP problem, a resilient robot team is expected to autonomously re-organize the active robots in an optimal manner to complete the unfinished tasks of failed robots.

Secondly, it is also important that the robot team operates efficiently. Typically, due to incorrect, incomplete or lack of a priori knowledge of the environment, the initial task allocation may be sub-optimal. As a result, some robots may finish their tasks earlier and become idle, which is a waste of their resources. Thus, it is critical that the robot team autonomously reallocates these idling robots in an optimal manner to assist other robots to reduce the total coverage time.

Fig. 1 illustrates the above concepts of resilience and efficiency. Fig. 1a shows an example of resilience where the neighbors of a failed robot proactively negotiate to decide whether any of them should leave its current task to fill the coverage gap. Fig. 1b shows an example of efficiency where a group of robots that have finished (or are close to finish) their current tasks negotiate to optimally reallocate to new tasks or to help other robots in their existing tasks.

1-B Challenges

The challenges associated with the problem of resilient and efficient MCPP are presented below.

• •

Scalability: The MCPP algorithm should be scalable to accommodate a growing number of tasks and/or robots, thus making a distributed control structure appropriate.

• •

Optimization factors: The optimization for task reallocation must consider the following factors:

Task worths, which can be quantified by the expected number of undiscovered targets (e.g., crops to cut or mines to discover) in the tasks. 2. 2.

Probabilities of success of the available robots in finishing the contested tasks, which depend on various factors including their current energy levels, the costs of traveling to the contested tasks, and the costs of finishing those tasks.

• •

Dynamically changing conditions: The conditions of robots as well as tasks change dynamically during coverage. The task worths decrease as targets are discovered. On the other hand, the robots drain their batteries during exploration, hence decreasing their probabilities of success. Therefore, the optimization process must accommodate these dynamic factors.

• •

Computation time: First of all, the optimization must be event-driven, i.e., triggered only in case of failures and/or idling. Secondly, once the optimization is triggered, the task reallocation decision must be made in a timely manner to avoid prolonged coverage time, thus motivating a local distributed event-focused optimization over only a subset of available robots and tasks.

• •

Connection between local and global objectives: Although the local optimization decision can be sub-optimal for the whole team, it is important that it is still aligned with the global objectives of the team. In other words, the local optimization must not only benefit the involved robots but also the whole team. The objectives include early detection of remaining targets, reduction in the total coverage time, and complete coverage.

• •

Complete coverage: The MCPP algorithm must guarantee complete coverage of the a priori unknown environment.

1-C Our Contributions

To the best of our knowledge, the concept of resilient coverage has not been adequately addressed in the existing MCPP methods. Thus, we present a novel online MCPP algorithm for resilient and efficient coverage in a priori unknown environments, which addresses the challenges discussed above. The algorithm is called Cooperative Autonomy for Resilience and Efficiency (CARE). For coverage control in each task, CARE utilizes our previously published $\epsilon^{\star}$ algorithm [1], which is a single-robot online CPP algorithm. More details of the $\epsilon^{\star}$ algorithm are provided in Section 2.

The CARE algorithm operates in a distributed yet cooperative fashion. Each robot is controlled using a Discrete Event Supervisor (DES), which triggers games between a set of feasible players in the event of a robot failure or idling, to make collaborative decisions for task reallocations. The game-theoretic structure is modeled using Potential Games [12], where the utility of each player is connected with a shared objective function for all players. In case of no failures, CARE reallocates idling robots to support other robots in their tasks; hence reduces coverage time and improves team efficiency. In case of robot failures, CARE guarantees complete coverage via filling coverage gaps by optimal reallocation of other robots; hence providing resilience, albeit with a possibly small degradation in coverage time.

The CARE algorithm has been validated on a high-fidelity robot simulator (i.e., Player/Stage) in various complex obstacle-rich scenarios. The results demonstrate that the team achieves complete coverage under failures and enables faster target discovery as compared to three alternative methods.

The rest of the paper is organized as follows. Section 2 presents a brief review of the existing MCPP algorithms. Section 3 formulates the MCPP problem and Section 4 presents the details of the CARE algorithm. The results are discussed in Section 5 and the paper is concluded in Section 6 with recommendations for future work.

2 Related Work

Existing CPP methods can be categorized as offline or online (i.e., sensor-based). Offline approaches assume the environment to be a priori known, while online approaches generate coverage paths in situ based on sensor information. Independently, CPP approaches are also described as randomized or systematic. Random strategies follow simple behavior-based rules, requiring in some cases no localization system or costly computational resources. However, they tend to generate strongly overlapped trajectories, thus unsuitable for time-critical operations. In contrast, systematic approaches are typically based on cellular decomposition [13][14] of the search area into sub-regions, and then adopt certain pre-defined path pattern (e.g., back and forth) for coverage in each sub-region; or by partitioning the search area into grid cells and then constructing potential field [15] or spanning trees [16][17] to generate coverage paths. In our previous work [1], we presented the $\epsilon^{\star}$ algorithm for single-robot CPP in unknown environments. The $\epsilon^{\star}$ algorithm uses an Exploratory Turing Machine (ETM) that consists of a 2D multilevel tape formed by Multiscale Adaptive Potential Surfaces (MAPS). The ETM stores and updates the explored, unexplored, and obstacle regions as time-varying potentials on MAPS. By default, the ETM adopts the lowest level of MAPS to generate the next waypoint; while it switches to higher levels as needed to evacuate from a local extremum. It is shown that $\epsilon^{\star}$ is computationally efficient for real-time applications, guarantees complete coverage in unknown environments, and does not require cellular decomposition of search area using critical points on obstacles. In CARE, $\epsilon^{\star}$ is used as the baseline coverage method by each robot to search within its task.

In terms of MCPP, Batalin and Sukhatme [20] proposed two local approaches for MCPP in unknown environments, based on mutually dispersive interaction between robots. Latimer et al. [21] presented a boustrophedon cellular decomposition-based approach using a team of circular robots. The robots operate together, but can split up into smaller teams when cells are created or completed. Rekleitis et al. [22] presented a distributed auction-based cooperative coverage algorithm, where the whole space is partitioned into tasks of fixed height and width, and robots utilized the Morse decomposition based single-robot CPP algorithm to search within each task. Sheng et al. [23] proposed a multi-robot area exploration method with limited communication range, where the waypoint of each robot is computed using a distributed bidding mechanism based on frontier cells. The bids rely on the information gain, communication limitation, and traveling costs to frontier cells. Rutishauser et al. [24] presented a distributed coverage method using miniature robots that are subject to sensor and actuator noise. Xu and Stentz [25] presented the $k$-Rural Postman Problem ($k$-RPP) algorithm to achieve environmental coverage with incomplete prior information using $k$ robots, that seeks to equalize the lengths of $k$ paths. Bhattacharya et al. [26] generalized the control law towards minimizing the coverage functional to non-Euclidean spaces, and presented a discrete implementation using graph search-based algorithms for MCPP. Karapetyan et al. [27] presented two approximation heuristics for MCPP in known environments, where the search area is divided into equal regions and exact cellular decomposition based coverage was used to search each region. Later, these methods were improved to consider vehicle kinematic constraints [28]. Yang et al. [29] proposed an online neural network based MCPP approach. In their method, the discovered environment was represented according to the dynamic activity landscape of the neural network, which is used to compute robot waypoints; and robots treat each other as moving obstacles during operation.

However, the above-mentioned algorithms have not addressed the problem of resilience in MCPP. In this regard, Agmon et al. [19] presented a family of Multi-robot Spanning Tree Coverage (MSTC) algorithms, where the Online Robust MSTC (ORMSTC) algorithm enables each robot to incrementally construct a local spanning tree to cover a portion of the whole space. If some robot fails, its local tree is released and taken over by its neighbors, but the already explored region of the failed robot must be scanned again. Also, the tree grows on the scale of $2\times 2$ cells, while if any cell within such larger cell is occupied by obstacles, the whole larger cell would not be covered, thus leading to incomplete coverage. Zheng et al. [30] presented a polynomial-time Multi-Robot Forest Coverage (MFC) algorithm that computes tree covers for each robot with trees of balanced weights, and they showed the superiority of MFC in time to MSTC via simulations; however, their algorithm does not consider failures. Song et. al. [9] presented the First-Responder (FR) cooperative coverage strategy, where early completed robots are reassigned to available new tasks that can maximize their own utility. However, this algorithm is not proactive, i.e., the coverage gaps caused by robot failures will not be filled until some other robots complete their tasks. Ferranti et. al. [18] presented the Brick and Mortar (B&M) algorithm, where the waypoint of each robot is computed locally based on the states of cells in the neighborhood. The idea behind B&M is to gradually thicken the blocks of inaccessible cells (i.e., visited or wall cells), while maintaining the connectivity of accessible cells (i.e., explored or unexplored cells). An unexplored cell can be marked as explored or visited, where the latter is allowed if it does not block the path between any two accessible cells in the neighborhood. The waypoint gives priority to the unexplored cell in the neighborhood, which has the most inaccessible cells around it. When some robot fails, the remaining robots continue regularly and the coverage gap becomes an extra workload; however, their method may produce redundant coverage due to the looping problem.

Research Gap: Although resilience concepts have been discussed in robot design [31], robot damage detection and recovery [32], flocking of robot teams [33] and networked control systems security under attacks [34], there is a scarcity of efforts that deal with the resilient coverage using multiple robots. Some of above-mentioned papers considered robot failures during coverage, however, their remedy was to simply release the coverage gaps to the remaining team, without optimization over the criticality (i.e., available worth) of such coverage gaps and the reliability of remaining robots. Thus, they are not proactive in filling the coverage gaps immediately if they satisfy optimization criteria, they wait until some other robots finish their tasks. In this regard, this paper presents a game-theoretic method for resilient and efficient coverage that incorporates these optimization factors while making event-driven proactive task reallocations. Table I presents a comparison of the key features of the CARE algorithm with the other relevant online MCPP algorithms.

3 Problem Description

This section presents the description of the robots, the MCPP problem and the performance metrics.

3-A Description of the Robots

Let $V=\{v_{1},v_{2},\ldots v_{N}\}$ be the team of $N\in\mathbb{N}^{+}$ robots, which are unmanned autonomous vehicles, as shown in Fig. 2. It is assumed that each robot is equipped with:

• •

a localization device (e.g., GPS) or a SLAM [35] system for operations in GPS-denied environments;

• •

a range detector (e.g., a laser) to detect obstacles within a radius $r_{s}\in\mathbb{R}^{+}$;

• •

a task specific sensor for performing the desired task; and

• •

a wireless communication device for (periodic or event-driven) information exchange between all pairs of robots. The communication is assumed to be perfect.

The robots continuously deplete the energies from their batteries; thus, their reliability is assessed based on the remaining energy as presented below.

Battery Reliability: Each robot $v_{\ell}\in V$, is assumed to carry a battery whose reliability [36], denoted as $R_{v_{\ell}}(t)$, can be computed as $R_{v_{\ell}}(t)=1-F(t)$, where $F(t)$ is the probability of battery being drained up to time $t$. Typically, the state-of-charge of a battery can be model using the realistic Kinetic Battery Model (KiBaM), which takes into account many important non-linear properties of batteries such as the rate-capacity effect and the recovery effect [37]. It is shown in [38] that with KiBaM, $F(t)$ follows a S-shaped curve when operating under different stochastic workload models (e.g., the on/off model and the burst model). The S-shaped curve can be approximated using a sigmoid function [36]. As such, the reliability of a robot $v_{\ell}$ is given as:

[TABLE]

where $\rho_{0}$ and $\rho_{1}$ indicate the curvature of the growth part and the inflection point, respectively. Their exact values depend on the choice of batteries. More details on the selection of these parameters are presented in Section 5.

3-B The MCPP Problem

The search area $\mathcal{R}\subset\mathbb{R}^{2}$ is assumed to be a planar field whose borderline is defined either by a hard barrier (e.g., walls or obstacles) or by a soft boundary (e.g., sub-area of a large field). A finite but unknown number of obstacles with arbitrary shapes are assumed to populate this area, but their exact locations and shapes are a priori unknown.

For the purpose of coverage path planning, a tiling $\mathcal{T}=\{\tau_{\alpha}\subset\mathbb{R}^{2},\alpha=1,\ldots|\mathcal{T}|\}$ is constructed to cover $\mathcal{R}$, i.e. $\mathcal{R}\subseteq\bigcup_{\alpha=1}^{|\mathcal{T}|}\tau_{\alpha}$, as shown in Fig. 2. Each $\tau_{\alpha}\in\mathcal{T}$ is called an $\epsilon$-cell, which is a square-shaped cell of side length $\epsilon\in\mathbb{R}^{+}$. The tiling is formed as minimal such that all $\epsilon$-cells are disjoint from each other, i.e., $\tau^{\circ}_{\alpha}\bigcap\tau^{\circ}_{\beta}=\emptyset$, $\forall\alpha,\beta\in\{1,\ldots|\mathcal{T}|\},\alpha\neq\beta$, where ∘ denotes the interior; and the removal of any single $\epsilon$-cell from $\mathcal{T}$ will destroy the covering.

The tiling $\mathcal{T}$ is partitioned into three sets: i) obstacle ($\mathcal{T}^{o}$), ii) forbidden ($\mathcal{T}^{f}$), and iii) allowed ($\mathcal{T}^{a}$). While the cells in $\mathcal{T}^{o}$ are occupied by obstacles, the cells in $\mathcal{T}^{f}$ create a buffer around the obstacles to prevent collisions due to inertia or large turning radius of the robots. Due to lack of a priori knowledge of the environment, the obstacle cells and forbidden cells are discovered online using sensor measurements. The remaining cells are allowed, which form the free space $\mathcal{R}^{a}=\bigcup_{\tau_{\alpha}\in\mathcal{T}^{a}}\tau_{\alpha}$ that is desired to be covered.

For distribution of multiple robots, an initial task allocation is required. Thus, the tiling $\mathcal{T}$ is grouped into $M$ disjoint regions $\{\mathcal{R}_{r}\subset\mathcal{T}:r=1,\ldots M\}$, s.t. $\mathcal{R}=\bigcup_{r=1}^{M}\mathcal{R}_{r}$. Each region $\mathcal{R}_{r}$ is regarded as one task and is referred as task $r$. Fig. 2 shows an example of the area with $M=3$ tasks. Each robot can work on one task at a time, but one task can be assigned to multiple robots. Note that $M$ may not be equal to $N$.

Remark 3.1**.**

The problem of optimal space partitioning into disjoint tasks and optimal initial robot allocations may require consideration of several factors (e.g., obstacle distribution, robot capabilities, and terrain types (or bathymetry)) and is beyond the scope of this paper. Here, we assume that no a priori knowledge of the environment is available, thus the tasks are made of equal sizes. However, as more information is obtained during exploration, event-driven task re-allocations are performed for performance improvement.

Definition 3.1** (Complete Coverage).**

Let $\epsilon_{\ell}(k)\in\mathcal{T}$ be the $\epsilon$-cell that is visited and explored by the robot $v_{\ell}$ at time $k$. Then the robot team $V$ is said to achieve complete coverage, if $\exists K\in\mathbb{N}$, s.t. the sequences $\{\epsilon_{\ell}(k),k=0,\ldots K\},\forall\ell=1,\ldots N$, jointly cover the free space $\mathcal{R}^{a}$, i.e.,

[TABLE]

In other words, the coverage is said to be complete if every cell in $\mathcal{R}^{a}$ is explored by at least one robot.

Next, it is assumed that each task contains randomly distributed targets, and their exact numbers and locations are unknown (details in Section 4-B). However, it is assumed that the expected number of targets in each task is known, which in practice could be obtained by various means such as field surveys, aerial views or prior knowledge from other sources.

Remark 3.2**.**

If the total number and spatial distribution of targets is known a priori, then complete coverage may not be necessary and an optimal traversing strategy could be constructed to find all the targets. However, in this paper, we assume that the planner neither knows the exact number of these targets, nor their exact locations, thus complete coverage becomes mandatory to guarantee finding all the targets.

Due to non-uniform spatial distribution of targets and obstacles within each task, the targets are discovered at unequal rates by all robots. Thus, at any point of time all tasks could contain significantly different numbers of undiscovered targets. It is therefore critical that the regions with the maximum number of targets are scanned earlier and are given priority. Early detection of targets helps when the mission is terminated prematurely due to emergencies, failures or other reasons and also provides mental comfort to the operator. For example, once the highly utilized areas of a building floor are cleaned then other areas could be cleaned gradually at ease.

Furthermore, the robots may suffer from unexpected failures during the coverage operation due to several reasons (e.g., sensor (or actuator) malfunctions, and mechanical defects) which lead to coverage gaps. Thus, it is important to fill these coverage gaps by task reallocations of healthy robots. It is also important that the criticality of the task of the failed robot, as measured by the expected number of remaining targets, is evaluated for task-reallocations in comparison with the existing tasks of healthy robots.

3-C Performance Metrics

The quality of multi-robot coverage can be evaluated based on the following performance metrics:

• •

Coverage ratio ($CR$): The ratio of the explored free space to the total free space, i.e.,

[TABLE]

Note that $CR<1$ if the coverage gaps caused by robot failures are left unattended.

• •

Coverage time ($CT$): The total operation time of the team. This is measured by the last robot that finishes its task.

• •

Remaining reliability ($RR$): The remaining reliability of all live robots at the end of the operation.

• •

Number of Targets Found ($NoTF$): The total number of targets discovered by the whole team.

• •

Time of Target Discovery ($ToTD$): The time for the whole team to discover a certain percentage of all targets. Note that the time of discovering all targets is less than or equal to the coverage time. Only in the limiting case, when the last target is discovered in the last visited cell by the robot that stops last, the coverage time will be equal to the $ToTD$ for all targets.

The objective of MCPP is to achieve $CR=1$ (even under a few robot failures), while minimizing $CT$, minimizing $ToTD$, and maximizing $RR$.

4 CARE Algorithm

The CARE algorithm addresses the above-mentioned MCPP problem via facilitating distributed event-driven task reallocations. In CARE, a set of local robots jointly re-plan their task assignments in two situations: (1) when a robot has finished its current task, or (2) when a robot has failed and is detected as non-responsive. The replanning algorithm employs a game-theoretic formulation, which computes the task worths and the success probabilities for each participating robot-task pair as optimization factors for optimal task reallocations. The task worths are measured by their expected number of undiscovered targets, while the probabilities of success of robot-task pairs are computed based on the robots’ battery reliabilities, travel times, and predicted times to finish the contested tasks.

CARE utilizes a distributed yet cooperative control architecture, where each robot $v_{\ell}\in V$ is controlled by a Discrete Event Supervisor (DES) that is modeled as a finite state automaton [39].

4-A Discrete Event Supervisor

The DES as shown in Fig. 3 is defined below.

Definition 4.1** (DES).**

The DES, denoted as $H$, is a deterministic finite state automaton represented by a $5$-tuple as follows

[TABLE]

where:

• •

$X=\{ST,WK,NG,RG,ID,FL,SP\}$* is the set of states, where $ST\equiv$ ‘Start’, $WK\equiv$ ‘Working’, $NG\equiv$ ‘No-idling Game’, $RG\equiv$ ‘Resilience Game’, $ID\equiv$ ‘Idle’, $FL\equiv$ ‘Failed’ and $SP\equiv$ ‘Stop’.*

• •

$\mathscr{E}=\{e_{0},e_{1},\ldots e_{7}\}$* is the finite set of events.*

• •

$\delta:X\times\mathscr{E}\rightarrow X$* is the partial state transition function. It is defined from one state to another if and only if there exists an arrow connecting them carrying an event.*

• •

$x_{0}=ST$* is the initial state.*

• •

$X_{m}=\{SP,FL\}$* is the set of marked states, which means a robot can either stop after finishing all the tasks or it may fail unexpectedly.*

While the states $ST$, $ID$, $FL$ and $SP$ are self-explanatory, the operations in states $WK$, $NG$ and $RG$ are described as follows. In state $WK$, the supervisor $H$ of robot $v_{\ell}$ adopts the $\epsilon^{\star}$ algorithm [1] for online coverage within its own task. Since no a priori information is available, all cells are initialized as unexplored. As the robot explores its task, it updates these cells as explored, obstacles and forbidden as suitable to track the progress of exploration [40]. This information is then periodically shared and synchronized with other robots such that each robot maintains a symbolic map of the entire region.

In states $RG$ and $NG$, $H$ triggers the Optimizer to play resilience games and no-idling games, respectively. The objective of resilience games is to optimally re-organize the neighbors of the failed robot to immediately fill the coverage gap, if it contains higher worth; while for no-idling games, the objective is to optimally reallocate the idling robot and its near-finishing neighbors to help other robots to reduce coverage time and collect more worth early. Details of Optimizer functionality are explained later in Section 4-B.

Events and State Transitions: The events in $\mathscr{E}$ enable state transitions in $H$, which are explained below. First, we define:

$r_{c}:V\rightarrow\{1,\ldots M\}$ to be the allocation function that indicates the current task allocations of robots; 2. 2.

$t_{c}:\{1,\ldots M\}\rightarrow[0,\infty)$ to be the remaining time required to complete a given task by its assigned robots; 3. 3.

$n_{U}:\{1,\ldots M\}\rightarrow\mathbb{N}$ to be the number of unexplored cells in a given task.

Now, consider a robot $v_{\ell}\in V$ that is currently working in task $r_{c}(v_{\ell})$. Event $e_{0}$ is generated when $v_{\ell}$ is turned on, and $H$ moves to the state $WK$ to start searching in task $r_{c}(v_{\ell})$ using the $\epsilon^{\star}$ algorithm.

Event $e_{1}$ is produced if any of its neighbor robot fails. This transitions $H$ to the state $RG$, that in turn invokes the Optimizer to play the resilience game to generate a task reallocation decision for $v_{\ell}$. Failure of a robot is detected using a standard mechanism based on heartbeat signals [41]. Each robot periodically broadcasts heartbeat signals, and also listening from others. Then a neighbor robot is detected as failed if its message is not received by $v_{\ell}$ constantly for a certain period of time $T_{0}\in\mathbb{R}^{+}$. To ensure robustness to false alarms, its failure is further confirmed if the majority of $\kappa_{2}\in\mathbb{N}^{+}$ neighbors detect its failure. Event $e_{2}$ occurs as soon as task $r_{c}(v_{\ell})$ is completed, i.e., the number of unexplored cells in task $r_{c}(v_{\ell})$, denoted as $n_{U}(r_{c}(v_{\ell}))$, becomes [math]. Event $e_{2}$ moves $H$ to the state $NG$, where the Optimizer is called to play the no-idling game for finding a new task for $v_{\ell}$.

Event $e_{3}$ appears if the Optimizer assigns a new (or current) task to robot $v_{\ell}$, which drives $H$ back to the state $WK$ to search in the assigned task; otherwise if no task is assigned, event $e_{4}$ is generated that moves $H$ to the state $ID$ and the robot becomes idle. Event $e_{5}$ is produced if some neighbor robot just completed its task and triggered the no-idling game, while $v_{\ell}$ is close to finish task $r_{c}(v_{\ell})$, i.e., $t_{c}(r_{c}(v_{\ell}))\leq\eta\in\mathbb{R}^{+}$, hence ready to reallocate after finishing the current task. Specifically, $t_{c}(r_{c}(v_{\ell}))=\frac{n_{U}(r_{c}(v_{\ell}))}{\omega}$, where $\omega\in\mathbb{R}^{+}$ is the speed of tasking a cell by the assigned robots. Then again, $H$ comes to the state $NG$ and the Optimizer is invoked to compute for a new task.

Event $e_{6}$ occurs when the entire area $\mathcal{R}$ is covered, by satisfying Eq. (2). This happens when no unexplored cells are left in the whole region, i.e., $\sum_{r=1}^{M}n_{U}(r)=0$. This moves $H$ to the terminal state $SP$ and the coverage is complete. At last, event $e_{7}$ is generated if $v_{\ell}$ itself is diagnosed as failed by its own diagnosis, and $H$ moves to the state $FL$. An advanced failure diagnostic tool is beyond the scope of this paper.

4-B Distributed Optimizer

The Optimizer is invoked by the supervisor $H$ to compute reallocation decisions under two conditions: (i) $H$ reaches $RG$ state upon detection of a neighbor failure (i.e., event $e_{1}$); or (ii) $H$ reaches $NG$ state upon completion of its own task (i.e., event $e_{2}$), or completion of a neighbor’s task (i.e., event $e_{5}$).

Specifically, the Optimizer is built based on the concept of Potential Games [12], which have the following advantages: (i) at least one Nash Equilibrium is guaranteed to exist, (ii) several learning algorithms are available (e.g., the Max-Logit algorithm [42][43]) that can converge fast to the optimal equilibrium, and (iii) the utility of each player is perfectly aligned with a globally shared potential function, thus as each player seeks to increase its own utility, the potential function is simultaneously improved and maximized upon reaching the optimal equilibrium.

Before presenting the details of the Optimizer, we list the various useful parameters in Table II. Some mathematical preliminaries are presented below.

Preliminaries: A game $G$ in strategic form [44] consists of:

• •

A finite set of players $\mathscr{P}=\{\mathscr{P}_{i}\in V:i=1,\ldots|\mathscr{P}|\}$, which includes all available robots that could be reallocated.

• •

A non-empty set of actions $\mathscr{A}_{i}$ associated to each player $\mathscr{P}_{i}$. In this paper, each action $a_{i}\in\mathscr{A}_{i}$ corresponds to the index of an available task, and the action set is assumed identical for all players, i.e., $\mathscr{A}_{i}=\mathscr{A}_{j}=\tilde{\mathscr{A}}$, $\forall i,j\in\{1,\ldots|\mathscr{P}|\}$.

• •

The utility function associated with each player $\mathscr{P}_{i}$, defined as $\mathscr{U}_{i}:\mathscr{A}_{\mathscr{P}}\rightarrow\mathbb{R}$, where $\mathscr{A}_{\mathscr{P}}=\mathscr{A}_{1}\times\ldots\times\mathscr{A}_{|\mathscr{P}|}$ denotes the set of joint actions for all players.

The utility function computes the payoff that $\mathscr{P}_{i}$ can receive by taking an action $a_{i}\in\mathscr{A}_{i}$, given that the rest of the players jointly select $a_{-i}\in\mathscr{A}_{-i}$, where $\mathscr{A}_{-i}:=\mathscr{A}_{1}\times\ldots\times\mathscr{A}_{i-1}\times\mathscr{A}_{i+1}\times\ldots\times\mathscr{A}_{|\mathscr{P}|}$. A joint action of all players $a_{\mathscr{P}}\in\mathscr{A}_{\mathscr{P}}$ is often written as $a_{\mathscr{P}}=(a_{i},a_{-i})$.

Definition 4.2** (Nash Equlibrium).**

A joint action $a_{\mathscr{P}}^{\star}=(a_{i}^{\star},a_{-i}^{\star})\in\mathscr{A}_{\mathscr{P}}$ is called a pure Nash Equilibrium if

[TABLE]

Definition 4.3** (Potential Games).**

A game $G$ in strategic form with action sets $\{\mathscr{A}_{i}\}_{i=1}^{|\mathscr{P}|}$ together with utility functions $\{\mathscr{U}_{i}\}_{i=1}^{|\mathscr{P}|}$ is a potential game if and only if, a potential function $\phi:\mathscr{A}_{\mathscr{P}}\rightarrow\mathbb{R}$ exists, s.t. $\forall$ $\mathscr{P}_{i}\in\mathscr{P}$

[TABLE]

$\forall$* $a_{i}^{\prime},a_{i}^{\prime\prime}\in\mathscr{A}_{i}$ and $\forall$ $a_{-i}\in\mathscr{A}_{-i}$.*

A potential game requires perfect alignment between the utility of an individual player and the globally shared potential function $\phi$ for all players, in the sense that the utility change by unilaterally deviating a player’s action is equal to the amount of change in the potential function. In other words, the potential function $\phi$ can track the changes in payoffs as some player unilaterally deviates from its current action. Therefore, if $\phi$ is designed as the global objective, then as players negotiate towards maximizing their individual utilities, the global objective is simultaneously optimized.

Now, let us present the resilience games and no-idling games modeled as potential games.

Specifics of Resilience Games and No-idling Games: Due to different objectives and triggering conditions, the player set and action set are fundamentally different for resilience games and no-idling games. Let $\mathscr{N}_{\kappa}^{v_{\ell}}$ denote the set of $\kappa\in\mathbb{N}^{+}$ nearest neighbors of robot $v_{\ell}$.

$\bullet$ No-idling Game: A no-idling game is triggered when some robot $v_{id}\in V$ completes its current task and becomes idle. Then, it calls its $\kappa_{1}$ nearest neighbors $v_{\ell}\in\mathscr{N}_{\kappa_{1}}^{v_{id}}$ that are close to finish their tasks to participate in the game. Thus, a no-idling game comprises of:

• •

$\mathscr{P}=\{v_{id}\}\cup\{v_{\ell}\in\mathscr{N}_{\kappa_{1}}^{v_{id}}:t_{c}(r_{c}(v_{\ell}))\leq\eta\}$.

• •

$\tilde{\mathscr{A}}=\{r\in\{1,\ldots M\}:t_{c}(r)\geq\gamma\in\mathbb{R}^{+}\}$, which contains incomplete tasks that have sufficient work left to be finished by their currently assigned robots. If some players still have some work left in their current tasks, they are assigned such that they finish their current tasks before being reallocated to new tasks.

$\bullet$ Resilience Game: A resilience game is triggered when some robot $v_{f}$ fails. Then, the $\kappa_{2}$ nearest neighbors of $v_{f}$ are involved in the game to re-optimize their current task allocations. Thus, a resilience game comprises of:

• •

$\mathscr{P}=\mathscr{N}_{\kappa_{2}}^{v_{f}}$.

• •

$\tilde{\mathscr{A}}=\{r_{c}(v_{f})\}\cup\{r_{c}(v_{\ell}),v_{\ell}\in\mathscr{N}_{\kappa_{2}}^{v_{f}}:t_{c}(r_{c}(v_{\ell}))>\eta\}$, which contains the current tasks of all players and the failed robot. The condition $t_{c}(r_{c}(v_{\ell}))>\eta$ ensures that those tasks close to be finished will be completed by their currently assigned robot and hence not needed to be part of the game.

Remark 4.1**.**

If there exist other active robots working in the same task of the failed robot, then they will take over this task and no resilience game is triggered.

Remark 4.2**.**

When a game is initiated, the information is exchanged and synchronized between all players, including their locations, discovered environment maps, success probabilities and estimated task worths.

Although the game specifics are different for the resilience and no-idling games, they follow the same design of the potential function and utility function as follows.

Design of Potential Function for Task Reallocations: As explained earlier in Section 1-B, the players must analyze the following optimization factors during task reallocation:

Task worths, which can be quantified by the expected number of undiscovered targets in the tasks. 2. 2.

Probability of success of each player to finish a certain task, which depends on its current battery reliability, the cost of traveling to the new task, and the cost of finishing the new task.

Thus, the potential function $\phi$ for all players in the game is defined to be the total expected worth [45] obtained by choosing a joint action $a_{\mathscr{P}}\in\mathscr{A}_{\mathscr{P}}$, as follows.

[TABLE]

where $\{\mathscr{P}\}_{r}\triangleq\{\mathscr{P}_{i}\in\mathscr{P}:a_{i}=r\}$ denotes the subset of players that choose the same task $r\in\{1,\ldots M\}$ in the joint action $a_{\mathscr{P}}$; $w_{r}$ is the current available worth of task $r$; and $p_{r}(\mathscr{P}_{i})$ is the success probability of player $\mathscr{P}_{i}$ to finish task $r$. The term p(r):=1-\prod_{\mathscr{P}_{i}\in\{\mathscr{P}\}_{r}}\big{[}1-p_{r}(\mathscr{P}_{i})\big{]} is the joint success probability for all players to finish task $r$ together.

As exploration continues, the conditions of robots and tasks change dynamically. Thus, the success probability $p_{r}(\mathscr{P}_{i})$ and the task worth $w_{r}$ in Eq. (6) must be updated before a game is played.

Computation of Success Probability: The success probability $p_{r}(\mathscr{P}_{i})$ is evaluated online using Eq. (1) as follows.

[TABLE]

where $R_{\mathscr{P}_{i}}(\tilde{t})$ is the reliability of player $\mathscr{P}_{i}$ at time $\tilde{t}$, which is estimated as

[TABLE]

where $t_{k}$ is the total tasking time of $\mathscr{P}_{i}$ since the beginning until the game was initiated, $t_{tr}$ is the traveling time to task $r$, and $t_{r}$ is the estimated time to complete task $r$. Specifically, $t_{tr}=\frac{Dist(\mathscr{P}_{i},r)}{u}$, where $Dist(\mathscr{P}_{i},r)$ measures the distance between player $\mathscr{P}_{i}$’s current location and the centroid of task $r$, and $u\in\mathbb{R}^{+}$ is the robot’s traveling speed; and the time $t_{r}=\frac{n_{U}(r)}{\omega}$, where $\omega$ is the speed of tasking a cell by the assigned robots.

In addition, if a robot is selected as a player to find a new task but it still has a small portion left in its current task, then it would like to first finish this task before being reallocated to a new one. Hence, an extra term $t_{c}$ is included in Eq. (8) if the estimated time to complete the unfinished part of its current task $r_{c}(\mathscr{P}_{i})$ satisfies $t_{c}(r_{c}(\mathscr{P}_{i}))\leq\eta$.

Computation of Task Worths: The worth $w_{r}$ in Eq. (6) indicates the expected number of undiscovered targets in task $r$ that are available to the players. Let $\mathbf{x_{r}}$ be a random variable that denotes the total number of targets in task $r$, which is assumed to follow the Poisson distribution with parameter $\lambda_{r}$. Its probability mass function is given as:

[TABLE]

If $\xi$ targets have been already discovered in task $r$, then the estimated remaining number of targets, $\tilde{w_{r}}$, is computed as:

[TABLE]

By definition, Poisson distribution has mean $\lambda_{r}$, i.e., $\sum_{x=0}^{\infty}x\cdot e^{-\lambda_{r}}\cdot\frac{{\lambda_{r}}^{x}}{x!}=\lambda_{r}$. Also, one has $\sum_{x=0}^{\infty}e^{-\lambda_{r}}\cdot\frac{{\lambda_{r}}^{x}}{x!}=1$. Thus, $\tilde{w_{r}}$ is computed as:

[TABLE]

Next, we decide the portion of $\tilde{w_{r}}$ available to the players, i.e., $w_{r}$. Since task $r$ may contain some robots currently working there but are not participating in the task reallocation, i.e., they are not players, then if a player selects task $r$, it must work together with these existing robots. In turn, the maximum payoff a player could expect from task $r$ becomes less due to sharing with the existing robots. Let $\bar{\mathscr{P}}\triangleq V\setminus\mathscr{P}$ denote the subset of robots that are not players. Similarly, let $\{\bar{\mathscr{P}}\}_{r}$ be the set of non-player robots that are currently working in task $r$, which have a joint success probability $q(r)$ for task $r$, i.e., q(r)=1-\prod_{v_{\ell}\in\{\bar{\mathscr{P}}\}_{r}}\big{(}1-p_{r}(v_{\ell})\big{)}. Then $w_{r}$ is computed as:

[TABLE]

Utility Function of Each Player: In order to form a potential game, the utility function, together with the potential function defined in Eq. (6), must satisfy Eq. (5). Since the utility of a player also depends on the actions taken by the rest of the players, thus a rule is needed to distribute the total produced payoff among contributing players. In this regard, this paper adopts the concept of Marginal Contribution due to its low computation burden thus feasible for online decision-making [45].

Definition 4.4** (Marginal Contribution).**

The marginal contribution of player $\mathscr{P}_{i}$ in a joint action $a_{\mathscr{P}}=(a_{i},a_{-i})$ is

[TABLE]

where $\emptyset$ represents player $\mathscr{P}_{i}$’s null action, indicating no task is assigned to it.

The utility function is derived as follows. First, substitute Eq. (6) into Eq. (12), one has:

[TABLE]

Note that for any task $r$ not selected by player $\mathscr{P}_{i}$, i.e., $r\neq a_{i}$, one has $\{\mathscr{P}\}_{r}=\{\mathscr{P}\}_{r}\setminus\mathscr{P}_{i}$. Thus, the produced potentials in these tasks are canceled in the above equation. It can then be further simplified as below, where $w_{a_{i}}$ is the worth of task $a_{i}$.

[TABLE]

Proposition 4.1**.**

The game $G$ with potential function $\phi$ of Eq. (6) and the utility function $\mathscr{U}_{i}$ of Eq. (4-B) is a potential game.

Proof.

Given a joint action $a_{-i}$, the difference in potential $\phi$ when player $\mathscr{P}_{i}$ deviates its action from $a_{i}^{\prime}$ to $a_{i}^{\prime\prime}$ is:

[TABLE]

Thus game $G$ satisfies Eq. (5) and it is a potential game. ∎

In this paper, the optimal equilibrium $a_{\mathscr{P}}^{\star}$ is acquired using the Max-Logit algorithm [42]. Before any game starts, each player computes its success probability $p_{r}(\mathscr{P}_{i}),\forall r\in\tilde{\mathscr{A}}$ using Eq. (7), and updates the estimated task worth $w_{r},\forall r\in\tilde{\mathscr{A}}$ using Eq. (10). Then, necessary information are communicated and synchronized as mentioned in Remark 4.2.

Algorithm 1 presents details to acquire $a_{\mathscr{P}}^{\star}$ in a distributed manner using Max-Logit. In particular, the initial joint action $a_{\mathscr{P}}(1)$ (line 1) is initialized as follows: for resilience games, $a_{i}(1)$ is set as the current task $r_{c}(\mathscr{P}_{i})$ of player $\mathscr{P}_{i}$, while for no-idling games, $a_{i}(1)$ is randomly picked from $\tilde{\mathscr{A}}$; then, $a_{\mathscr{P}}(1)$ is determined via synchronization with all other players.

Once $a_{\mathscr{P}}^{\star}$ is obtained using Algorithm 1, the new task $r$ for player $\mathscr{P}_{i}$ is set as its action $a_{i}^{\star}$ in the equilibrium $a_{\mathscr{P}}^{\star}$.

Post-game Coordination: If multiple robots (including both existing robots and incoming players) are assigned to the same task $r$, it becomes imperative to utilize some strategy to ensure their safety and efficiency when searching together. Let $n_{\max}\in\mathbb{N}^{+}$ be the maximum number of robots allowed to work in the same task at the same time. In this regard, task $r$ is evenly partitioned into $n_{\max}$ sub-regions, where each sub-region is only allowed one robot at a time.

Consider some non-player robot $v_{\ell}\in\{\bar{\mathscr{P}}\}_{r}$ that is currently working in task $r$. It continues as usual but its task is restricted to the sub-region determined by its current location. This produces $n_{0}\in\mathbb{N}$ incomplete sub-regions that are instantly available to the incoming players. Now consider a player $\mathscr{P}_{i}\in\{\mathscr{P}\}_{r}^{\star}\triangleq\{\mathscr{P}_{i}\in\mathscr{P}:a_{i}^{\star}=r\}$ that is also assigned to task $r$. It selects the sub-region by following the steps below. First, it computes its rank in $\{\mathscr{P}\}_{r}^{\star}$ based on its success probability. If it ranks in the top $n_{0}$ and all other players ranked above it have selected their new sub-region, then it selects the new available sub-region for itself that minimizes its traveling distance. However, if it ranks after $n_{0}$, it stays temporarily idle but can later be reactivated to replace any robot in task $r$ should it fail. Once $\mathscr{P}_{i}$ finds a new sub-region, its centroid is set as the movement goal. As described previously, $\mathscr{P}_{i}$ resumes to search its new sub-region using the $\epsilon^{\star}$ algorithm upon its arrival, and its supervisor $H$ transitions to the state $WK$ accordingly.

4-C Computational Complexity of the Optimizer

As described above, once the Optimizer triggers a game involving the player set $\mathscr{P}$ and the action set $\tilde{\mathscr{A}}$, the joint action $a_{\mathscr{P}}$ is first initialized locally and then synchronized with other players. This process takes $O(|\mathscr{P}|)$ complexity.

Thereafter, the game follows Algorithm 1 in a distributed manner, which operates in a loop for a user-defined $L\in\mathbb{N}^{+}$ computation cycles. At each cycle, one player is randomly selected and is allowed to probabilistically alter its action, which takes $O(|\mathscr{P}|)$ to find out if $\mathscr{P}_{i}$ is selected. If not, its action is repeated, which takes $O(1)$ complexity; otherwise, $\mathscr{P}_{i}$ first randomly chooses an alternative action $\hat{a_{i}}\in\tilde{\mathscr{A}}$ with equal probability, which is $O(|\tilde{\mathscr{A}|})$. Then the associated utility $\mathscr{U}_{i}(\hat{a_{i}},a_{-i})$ is computed using Eq. (4-B), which takes $O(|\mathscr{P}|)$ complexity. Thereafter, $\mathscr{P}_{i}$ uses $\hat{a_{i}}$ to update its action $a_{i}$ in a probabilistic manner, which has $O(1)$. At the end of each cycle, the updated action $a_{i}$ is transmitted to other players, which requires $O(|\mathscr{P}|)$ complexity.

Therefore, in the worst case where $\mathscr{P}_{i}$ is selected in every cycle, the total complexity becomes $O(|\mathscr{P}|+L\cdot(3|\mathscr{P}|+|\tilde{\mathscr{A}}|))$. In comparison, for a centralized optimization algorithm, it must search over $|\tilde{\mathscr{A}}|^{|\mathscr{P}|}$ possible joint actions, which grows significantly faster as $|\mathscr{A}|$ and $|\mathscr{P}|$ increase.

4-D Connection between Local Games and Team Potential

As discussed earlier, in both resilience games and no-idling games, the potential function $\phi$ is optimized for the set of players, which form a subset of the robot team. Now, we show that the increase in $\phi$ will directly improve the performance of the whole team.

To illustrate this, let $\Phi(a)$ denote the total team potential that defines the total expected worth achievable by the team, where $a=(a_{\mathscr{P}},a_{\bar{\mathscr{P}}})$ is the joint action of the team including players $\mathscr{P}$ and non-players $\bar{\mathscr{P}}$. Note for the non-players, the action $a_{\bar{\mathscr{P}}}$ simply represent their current tasks. Since the players and non-players are mixed and distributed over different tasks, the total team potential $\Phi(a)$ is defined as:

[TABLE]

where $\{V\}_{r}=\{\mathscr{P}\}_{r}\cup\{\bar{\mathscr{P}}\}_{r}$ is the set of all robots that are assigned to task $r$ in the joint action $a$, and the term within the parentheses on the right hand side computes the joint success probability to complete task $r$ by all of its assigned robots.

As the players reach the optimal equilibrium, the joint action becomes $a^{\star}=(a_{\mathscr{P}}^{\star},a_{\bar{\mathscr{P}}})$ and the team potential becomes $\Phi(a^{\star})$.

Theorem 4.1**.**

The optimal equilibrium $a^{\star}$ increases the total team potential $\Phi(a)$, i.e., $\Phi(a^{\star})\geq\Phi(a)$.

Proof.

First, let us show that the team potential $\Phi(a)$ is separable by the worth created by the players (i.e., $\mathscr{P}$) and the rest of the robots (i.e., $\bar{\mathscr{P}}$). Then we will investigate the change of $\Phi$ due to task reallocation. Now $\Phi(a)$ can be decomposed as follows.

[TABLE]

where $p(r):=1-\prod_{\mathscr{P}_{i}\in\{\mathscr{P}\}_{r}}[1-p_{r}(\mathscr{P}_{i})]$ and $q(r):=1-\prod_{v_{\ell}\in\{\bar{\mathscr{P}}\}_{r}}[1-p_{r}(v_{\ell})]$ are used to denote the joint success probability of the players and the rest of the robots that are assigned to task $r$ in the joint action $a$, respectively.

Then we can further break down $\Phi(a)$ as follows.

[TABLE]

where the second term in the last step is the worth generated by the players (if any) that would like to finish the small unfinished part in their current tasks before being reallocated to new tasks, while the third term indicates the worth generated by the non-player robots $\bar{\mathscr{P}}$. The values of both these terms do not change by games. Since $\phi(a_{\mathscr{P}}^{\star})\geq\phi(a_{\mathscr{P}})$, $\forall a_{\mathscr{P}}\in\mathscr{A}_{\mathscr{P}}$, we have $\Phi(a^{\star})\geq\Phi(a)$. ∎

Game Performance Metrics: The quality of the task reallocation decision (i.e., $a_{\mathscr{P}}^{\star}$ for the players and $a^{\star}$ for the team) can be evaluated by the worth gain. Note that in any task reallocation, there is a tradeoff between whether the robot should continue with its current task or reallocate to a new task. Thus, where a higher gain implies early detection of targets. Specifically, at player-level, the Gain of Players ($G_{P}$) is defined as

[TABLE]

Similarly, at team-level, the Gain of Team ($G_{T}$) is

[TABLE]

Note that since $\phi(a_{\mathscr{P}}^{\star})\geq\phi(a_{\mathscr{P}})$ and $\Phi(a^{\star})\geq\Phi(a)$, both $G_{p}$ and $G_{T}$ are non-negative, which implies that the outcome of a game results in the gain of worth not only for the players but also for the whole team. Both $G_{P}$ and $G_{T}$ will be quantitatively examined in Section 5.

4-E Complete Coverage under Failures

The success of finding all hidden targets relies on the complete coverage of the whole area $\mathcal{R}$. Due to the completeness of the underlying single-robot coverage algorithm [1], each task can be fully covered by the assigned robot in finite time if it stays alive. Now, let us examine coverage under failures.

Theorem 4.2**.**

The CARE algorithm guarantees complete coverage in finite time as long as one robot is alive.

Proof.

Consider a robot $v_{\ell}$ that is alive during the whole operation, whose supervisor $H$ starts with the state $WK$ upon robot being turned on. We show below that $v_{\ell}$ must reach the terminal state $SP$ in finite time, which happens if and only if $\sum_{r=1}^{M}n_{U}(r)=0$, i.e., complete coverage.

First, as shown in Fig. 3, any cycle between states in $H$ involves either state $NG$ or $RG$. Also, a robot can reach the states $NG$ or $RG$ due to completion of some task or failure of some robot, respectively. Now, since there are only a finite number of robots (i.e., $N$) and a finite number of tasks (i.e., $M$), each robot can visit these states only a finite number of times. Thus, $H$ cannot have any live lock. Moreover, in states $NG$ or $RG$, it takes a finite amount of time to reach an equilibrium solution using the Max-Logit algorithm. Thus, $H$ will always switch to either state $WK$ or $ID$ after games. In state $WK$, the underlying $\epsilon^{\star}$ algorithm is used to explore in the current assigned task $r_{c}(v_{\ell})$ of robot $v_{\ell}$. As shown in [1], $\epsilon^{\star}$ constantly reduces $n_{U}(r_{c}(v_{\ell}))$ until task $r_{c}(v_{\ell})$ is completed in finite time, so $H$ can only stay in state $WK$ for finite time.

Further, in state $ID$, $H$ can either be invoked to play new games and hence move to states $NG$ or $RG$, or it can move to the state $SP$ upon complete coverage, i.e., $\sum_{r=1}^{M}n_{U}(r)=0$. Since the former case can only happen for a finite number of times, $H$ will come back to the state $ID$ when no incomplete task is available to $v_{\ell}$ anymore. The same logic applies to all other active robots. Thus, all active robots including $v_{\ell}$ will reach state $ID$ in finite time, which implies that no incomplete tasks exist, i.e., $\sum_{r=1}^{M}n_{U}(r)=0$. Then, they all transition to the terminal state $SP$ and the complete coverage is achieved. ∎

5 Results and Discussion

The CARE algorithm was validated on the high-fidelity robotic simulation platform called Player/Stage [46] using a computer with $3.40$ GHZ CPU and $16$ GB RAM. The Player provides a software base whose libraries contain models of different types of robots, sensors and actuators. On the other hand, Stage is a highly configurable robot simulator.

In this section, we present the performance of the CARE algorithm in three complex obstacle-rich scenarios. The search area $\mathcal{R}$ of size $50m\times 50m$, was partitioned into a $50\times 50$ tiling consisting of $\epsilon$-cells of size $1m\times 1m$. The $\mathcal{R}$ was partitioned into $M=10$ tasks $\{\mathcal{R}_{r}:r=1,\ldots 10\}$, each of size $10m\times 25m$. Each task $r\in\{1,\ldots 10\}$ was initially assigned with one robot, and a maximum number of $n_{\max}=4$ robots were allowed to search together in one task. Each task $r$ contained an unknown number of targets distributed randomly according to the Poisson distribution with mean $\lambda_{r}\in\{1,\ldots 32\}$.

A team of $N=10$ Pioneer 2AT robots was simulated, where each robot has dimensions of $0.44m\times 0.38m\times 0.22m$, and was equipped with $16$-beam laser scanners with a detection range of $r_{s}=5m$. The kinematic constraints of the robot, such as the top speed of $0.4m/s$ and the minimum turn radius of $0.04m$, were included in the simulation model. The tasking speed was set as $\omega=0.32$cells/s. The parameters $\rho_{0}$ and $\rho_{1}$ in the battery reliability model are chosen such that each robot can finish one and two tasks with more than $0.9$ and $0.4$ remaining reliability, respectively. Specifically, based on the size of each task ($250$ cells) and the robot tasking speed, it takes $\sim 780s$ to finish an obstacle-free task. Then, using Eq. (1):

[TABLE]

which lead to $\rho_{0}\sim 3.0\times 10^{-3}$ and $\rho_{1}\sim 1400$s. Then, considering stochastic uncertainties in the initial battery charging conditions, these parameters are generated on different robots using Gaussian distributions, s.t., $\rho_{0}\sim N(3\times 10^{-3},7.5\times 10^{-5})$ and $\rho_{1}\sim N(1400,35)$, where the standard deviation is chosen as $2.5\%$ of the corresponding mean value.

Initially, due to lack of a priori knowledge of the environment, all $\epsilon$-cells are initialized as unexplored, and as the robot explores the environment, the obstacle and forbidden cells are discovered and updated accordingly. The game parameters are chosen as: $\kappa_{1}=6$ and $\kappa_{2}=3$, and in Max-Logit, the number of computation cycles is set as $L=50$ and learning parameter is $\tau=0.05$. The other parameters $\eta$ and $\gamma$ are chosen as follows. We set $\eta$ such that it corresponds to less than $4\%$ of the time to finish one task, i.e., $780\times 4\%=31.2s$. Thus, robots which have only $4\%$ of the task left will participate as players for no-idling games. Similarly, we set $\gamma$ such that it corresponds to over $25\%$ of the time to finish one task, i.e., $780\times 25\%=195s$. Thus, tasks which have still more than $25\%$ unexplored area become contested tasks. Hence, further rounding up we used $\eta=30s$ and $\gamma=200s$.

5-A Scenario 1: No Failures but Some Robots Idle

Fig. 4a presents the cooperative coverage of a complex islands scenario. A total number of $107$ targets were distributed randomly in the field. No failure appeared throughout the whole search, while two no-idling games were triggered to reallocate early completed robots to reduce the coverage time. Each subfigure in Fig. 4a, i.e., Fig. 4a(1)$\sim$4a(8), is comprised of a top figure showing the trajectories of robots by different colors, and a bottom figure showing the corresponding overall symbolic map of the entire search area $\mathcal{R}$, which is periodically synchronized and merged by all live robots. The different colors in the symbolic map represent the following regions: i) light green for obstacles, ii) medium green for unexplored, iii) dark green for explored with no obstacles, and iv) yellow for the forbidden region around the obstacles.

Fig. 4a$(1)$ shows that the robots started exploration and used their on-board sensing systems to explore the a priori unknown environment. Fig. 4a(2) shows that the robots continue searching within their assigned tasks. Fig. 4a$(3)$ shows the instance when robot $v_{10}$ finished task $10$ and triggered a no-idling game $G_{1}$. The player set was formed as $\mathscr{P}=\{v_{5},v_{10}\}$, where $v_{5}$ was near finishing its task. At that moment, tasks $3$, $4$, $7$ and $9$ still had a lot of area unexplored and required significant time to finish by their currently assigned robots, thus they formed the action set $\tilde{\mathscr{A}}=\{3,4,7,9\}$. The optimal equilibrium of $G_{1}$ reassigned $v_{5}$ and $v_{10}$ to task $4$ and task $9$, respectively. Because task $10$ was already completed, $v_{10}$ immediately traveled to its new task $9$, while $v_{5}$ had to first finish the remainder of its current task $5$ before moving to task $4$. Since there was a robot $v_{9}$ currently working in task $9$, the post-game coordination strategy was used to further partition task $9$ into $n_{\max}=4$ sub-regions. As observed in Fig. 4a$(4)$, $v_{10}$ selected the closest sub-region in the upper right corner and searched in parallel with $v_{9}$. Similar post-game coordination was performed when $v_{5}$ joined to search with $v_{4}$ in task $4$.

Later, another no-idling game $G_{2}$ was triggered when $v_{6}$ finished task $6$, as shown in Fig. 4a$(5)$. The player set was formed as $\mathscr{P}=\{v_{2},v_{6}\}$, where $v_{2}$ was near finishing its task. Since tasks $4$ and $9$ have been assigned with extra robots after game $G_{1}$, the estimated time to finish these tasks dropped significantly, hence they were excluded from game $G_{2}$. The tasks $3$ and $7$, however, still required significant time to finish, thus they formed the action set $\tilde{\mathscr{A}}=\{3,7\}$. The optimal equilibrium of $G_{2}$ in turn reassigned $v_{2}$ and $v_{6}$ to task $3$ and task $7$, respectively, as shown in Fig. 4a$(6)$. It is observed in Fig. 4a(7) that, robot $v_{2}$ selected the upper right sub-region of task $3$ and continued searching in parallel with robot $v_{3}$, while robot $v_{6}$ joined robot $v_{7}$ to search task $7$ in a similar fashion. Finally, complete coverage was achieved with all targets discovered, as shown in Fig. 4a(8).

Fig. 4b summarizes the specifics and performance of the two games. As observed, the player-level worth gain $G_{P}$ reached $54.23\%$ and $92.45\%$ in games $G_{1}$ and $G_{2}$, respectively, which means that after task reallocations, the idling robots can expect a higher number of targets from the remaining tasks. At the team-level, $G_{T}$ is $3.29\%$ for $G_{1}$ and $2.42\%$ for $G_{2}$, thus the whole team also benefits from the task reallocations.

5-B Scenario 2: Some Robots Fail and Some Idle

Fig. 5a presents a more complex scenario where two robots failed unexpectedly during operation. A total number of $106$ targets were randomly distributed in the field.

Fig. 5a(1) shows that the robots start exploration while using their on-board sensing systems to discover the environment. Fig. 5a$(2)$ shows that robot $v_{7}$ failed unexpectedly and a resilience game $G_{1}$ was triggered involving $\kappa_{2}=3$ of its closest neighbors. The player set was formed as $\mathscr{P}=\{v_{2},v_{6},v_{8}\}$. The action set consisted of the current tasks of all players, as well as the task belonging to the failed robot, i.e., $\tilde{\mathscr{A}}=\{2,6,7,8\}$. As seen in Fig. 5a(3), the optimal equilibrium of $G_{1}$ immediately reallocated $v_{8}$ to drop its current task and help $v_{7}$, because task $7$ has a much higher expected worth even at the expense of traveling. Later, as shown in Fig. 5a$(4)$, robot $v_{4}$ failed too, which initiated the second resilience game $G_{2}$. Similarly, robots $v_{3}$, $v_{5}$ and $v_{9}$ were the closest neighbors, hence they formed the player set $\mathscr{P}=\{v_{3},v_{5},v_{9}\}$. The action set was $\tilde{\mathscr{A}}=\{3,4,5,9\}$. As observed in Fig. 5a$(5)$, the optimal equilibrium of $G_{2}$ lead $v_{3}$ to drop its task $3$ and immediately transition to help $v_{4}$, in pursuit of a higher worth task. Since tasks $3$ and $8$ were dropped by their initially assigned robots after games $G_{1}$ and $G_{2}$, thus they are available to any future no-idling games. Fig. 5a$(6)$ shows that robot $v_{6}$ has just completed its task and triggered the third no-idling game $G_{3}$. It called the other robot $v_{1}$ to join $G_{3}$, which was close to finish task $1$. Thus the player set was $\mathscr{P}=\{v_{1},v_{6}\}$. The action set $\tilde{\mathscr{A}}=\{3,8\}$ included tasks $3$ and $8$ that were assigned with no robot. No other region had sufficient task left. The optimal equilibrium of $G_{3}$ reallocated $v_{1}$ and $v_{6}$ to task $3$ and task $8$, respectively, as seen in Fig. 5a$(7)$. Finally, complete coverage was achieved with all targets found, as shown in Fig. 5a(8).

Fig. 5b presents the details of all games. It is observed that $G_{P}$ is $37.64\%$ for $G_{1}$ and $36.98\%$ for $G_{2}$, thus via event-driven task reallocations, the neighbors of the failed robot can re-organize to compensate for the loss of expected worth due to robot failures. Also, $G_{P}$ is $91.16\%$ for game $G_{3}$, hence the idling robots can expect to discover more targets from the remaining tasks after task reallocation. Accordingly, $G_{T}$ is $15.22\%$, $16.19\%$ and $5.25\%$ for games $G_{1}$, $G_{2}$ and $G_{3}$, respectively. Thus, the whole team also benefits from each task reallocation.

5-C Performance Comparison with Alternative Methods

Now, we examine the performance of CARE as compared to three other online multi-robot coverage methods, including: (1) Non-cooperative (Non-Co.) strategy, where each robot covers its own task using the $\epsilon^{\star}$ algorithm without cooperation upon task completion or robot failures; (2) modified First-responder (FR) strategy [9], where robots that finish early would selfishly seek for new tasks that can maximize their own utility using Eq. (4-B); and (3) Brick and Mortar (B&M) algorithm [18]. The performance metrics of the alternative methods have been examined in both scenarios using the same initial conditions. The time measured in performance metrics is in seconds.

5-C1 Scenario 1

Figs. 6a$\sim$6d show the robot trajectories for Scenario $1$ using CARE and the three alternative methods. Since there was no failure in this scenario, complete coverage was achieved using all methods. The corresponding performance metrics are shown in Fig. 7.

As seen in Fig. 7a, CARE requires the least coverage time ($CT$), which saves $\frac{694.3-557.9}{694.3}\approx 19.65\%$ in time as compared to using the Non-Co method. Similarly, CARE saves about $16.41\%$ and $60.47\%$ in time as compared to using the FR and B&M methods, respectively. The significant savings in $CT$ are due to the no-idling games that reallocated early completed robots $v_{2}$, $v_{5}$, $v_{6}$, and $v_{10}$ in an optimized way.

Moreover, due to lack of cooperation, the Non-Co. method requires a much higher $CT$ than CARE and the FR method. In the FR strategy, since early completed robots selfishly selects their new tasks that can maximize their own utility, robot $v_{6}$ ended up picking task $3$ upon finishing task $6$, which contains higher worth even at the expense of long traveling time (see Fig. 6c). In this regard, the FR method presents higher $CT$ than CARE. Further, due to lack of task partitioning as well as the looping problem, the B&M method generated strongly overlapped trajectories that leads to the highest $CT$.

Fig. 7c shows the minimum, mean and maximum remaining reliability ($RR$) among all live robots as the operation ended. It is seen that, although CARE shares the same mean $RR$ with the Non-Co. and FR methods, it has the smallest difference between minimum and maximum $RR$ of all robots, which implies a more balanced battery depletion for different robots. In contrast, the B&M method presents the smallest mean $RR$ due to the highest $CT$.

As for the number of targets found ($NoTF$), since $CR=1$ in this scenario, all $107$ hidden targets were found using all methods, as shown in Fig. 7b$\sim$7d.

Fig. 7e shows the time of target discovery ($ToTD$). It is seen that at each percentage of targets found, CARE always requires the least time, thus leading to the fastest target discovery progress as compared to other methods. This is due to the optimized task reallocations of early completed robots $v_{2}$, $v_{5}$, $v_{6}$ and $v_{10}$ after playing no-idling games $G_{1}$ and $G_{2}$. Note that the time in $ToTD$ when $100\%$ targets are discovered, is different from $CT$, because robots should continue searching in unexplored regions towards complete coverage.

5-C2 Scenario 2

Figs. 8a$\sim$8d show the robot trajectories using CARE and the three alternative methods for Scenario $2$. The corresponding performance metrics are presented in Fig. 9. In this scenario, two robots of $v_{4}$ and $v_{7}$ failed during operation. The alternative methods were evaluated using the same failing condition, i.e., the same robots failed after traveling for the same amount of time.

As shown in Fig. 9a, CARE saves about $17.46\%$ and $44.35\%$ in $CT$ as compared to using the FR and B&M methods, respectively. This is due to the no-idling game $G_{3}$ that reallocated early completed robots $v_{1}$ and $v_{6}$ in an optimized manner; while in the FR method, the initially assigned tasks of the failed robots were left unattended until some other robot completes its task. Again, the B&M method has the highest $CT$ due to strongly overlapped trajectories.

Fig. 9c shows the $RR$ of live robots at the end of the team operation. It is seen that, CARE has a higher mean $RR$, as well as the smallest difference between minimum and maximum $RR$ of all live robots, as compared to the FR and B&M methods. Also, since tasks $4$ and $7$ were left unattended after robots $v_{4}$ and $v_{7}$ failed, the Non-Co. method has the highest mean $RR$.

As shown in Fig. 9b, coverage was incomplete ($CR=0.89$) using the Non-Co. method, while all other methods achieved $CR=1$. Accordingly, a total number of $28$ hidden targets were missed using the Non-Co. method, while all other methods successfully discovered all $106$ targets, as shown in Fig. 9d.

Fig. 9e shows the performance of $ToTD$, where CARE again shows the fastest target discovery progress as compared to other methods. This is mainly because after playing the resilience games $G_{1}$ and $G_{2}$, robots $v_{3}$ and $v_{6}$ immediately determined to drop their current tasks and search tasks $4$ and $7$ when robots $v_{4}$ and $v_{7}$ failed, respectively, in pursuit of a much higher expected worth than their current tasks. Also, the Non-Co. method only collected around $73.6\%$ of all targets at the end of the team operation due to incomplete coverage.

5-D Effects of Parameters on Coverage Performance

This section evaluates the effects of different parameters on the coverage performance. Specifically, we vary the values of $N$, $\lambda_{r}$, $\kappa_{1}$ and $\kappa_{2}$, while keeping the values of all other parameters constant. The performance metrics presented in Section 3-C are used for evaluations.

5-D1 Team Size ($N$)

We examine the effectiveness of CARE when different number of robots are deployed to search the same area $\mathcal{R}$. For this purpose, we present Scenario 3, where teams of $N=4$, $6$, $8$ and $10$ robots were deployed to cover the same $10$ tasks, as shown in Fig. 10. Fig. 10a, Fig. 10b, Fig. 10c and Fig. 10d present the coverage trajectories at different time instants for $N=4$, $6$, $8$, and $10$, respectively. The scenario setup is kept the same across all simulations, where robot $v_{4}$ fails after it travels for the same amount of time. As seen in Fig. 10a(1), Fig. 10b(1), Fig. 10c(1) and Fig. 10d(1), a resilience game was initiated upon failure of $v_{4}$, and its neighbor $v_{3}$ immediately dropped its task $3$ to help $v_{4}$, due to a higher expected worth. Later several no-idling reallocations occurred and in all cases complete coverage was achieved. Moreover, it is seen that with a smaller $N$, task reallocation appears more often. As shown in Fig. 10a(4), Fig. 10b(4), Fig. 10c(4) and Fig. 10d(4), the total coverage time clearly decreases when $N$ increases.

Table III presents the corresponding coverage performances. It is seen that with smaller $N$, since each robot must cover more tasks, the average $RR$ of all live robots become smaller. Also, the minimum and maximum $RR$ are close to the mean, which implies the live robots have been operating for similar amounts of time and robot idling was successfully prevented.

5-D2 Distribution of Targets ($\lambda_{r}$)

Here, we utilize a team of $8$ robots to examine the performance of CARE under different target distributions. Fig. 11 shows the coverage trajectories at different time instants under three different target distribution examples. The number of targets in each task $r$ is labeled in Fig. 11a(1), Fig. 11b(1), and Fig. 11c(1), and $\lambda_{r}$ is set as the actual number in each task. While the target distributions are randomly generated, in particular, the task of the failed robot $v_{4}$ has significantly different $\lambda_{r}$ in the three examples.

In target distribution example $1$, as seen in Fig. 11a, task $4$ has sparse targets. When $v_{4}$ failed, its neighbors $v_{2}$, $v_{3}$ and $v_{5}$ played a resilience game, but none of them was reallocated to help $v_{4}$. This is because they can expect higher utility from their current tasks at the moment. Later, as shown in Fig. 11a(3), when $v_{1}$ and $v_{5}$ finished, they were reallocated to tasks $6$ and $10$ after playing no-idling games, respectively. At that moment, due to much higher estimated worths in tasks $6$ and $10$, again none of them was reallocated to task $4$. At last, as seen in Fig. 11a(4), when $v_{2}$ and $v_{3}$ finished, they moved to task $4$ and eventually complete coverage was achieved and all targets were found.

Fig. 11b shows the coverage trajectories under target distribution example $2$. As compared to the previous example, now tasks $4$ and $5$ have slightly more targets, but less targets are present in tasks $3$. Thus, upon failure of $v_{4}$, $v_{3}$ was reallocated to task $4$ to pursue a higher utility, as shown in Fig. 11b(2). Later, multiple no-idling games were played and the idling robots $v_{2}$, $v_{7}$, $v_{8}$ and $v_{9}$ were reallocated, as shown in Fig. 11b(3). Finally, complete coverage was achieved as shown in Fig. 11b(4).

In target distribution example $3$, as shown in Fig. 11c(1), task $4$ has significantly more targets, which makes it prioritized for coverage. In contrast, tasks $3$ and $5$ have much less targets. Hence, as seen in Fig. 11c(2), when $v_{4}$ failed, a resilience game was initiated involving players $v_{2}$, $v_{3}$ and $v_{5}$; then both $v_{3}$ and $v_{5}$ moved to task $4$. Upon reaching task $4$, each of them picked a sub-area and worked in parallel. Thereafter, as shown in Fig. 11c(3), multiple no-idling games appeared and all the live robots were reallocated all around to fill the incomplete tasks. At the end, as shown in Fig. 11c(4), complete coverage was achieved with all targets found.

Based on the above analysis, it is seen that the target distribution has a direct impact on the game decisions, which tends to drive the players to pursue prioritized coverage in tasks with higher estimated worth in general.

5-D3 Player Set Size Parameters ($\kappa_{1},\kappa_{2}$)

Now, we examine the effects of neighborhood sizes $\kappa_{1}$ and $\kappa_{2}$ on the coverage performance for a team of $8$ robots in total. Specifically, we focus on two aspects: (1) using a varying $\kappa_{1}$ with a fixed $\kappa_{2}$; and (2) using a varying $\kappa_{2}$ with a fixed $\kappa_{1}$. The team-level performance metric $ToTD$ is used for evaluation.

First, we fix $\kappa_{2}=3$ and vary $\kappa_{1}$ in Scenario $1$. Note that $\kappa_{1}$ describes the neighborhood size in no-idling games, which is used to define player set $\mathscr{P}$ in Section 4-B. However, the players within the $\kappa_{1}$ neighborhood of the idling robot $v_{id}$, must also satisfy another condition of being close to finish their current tasks at that moment. Thus, the actual number of players (i.e., $|\mathscr{P}|$) could be smaller than $\kappa_{1}$.

Fig. LABEL:fig:vary_kappa_1 shows the $ToTD$ when $\kappa_{1}$ gradually increases from $2$ to $7$ in Scenario $1$. As described in Section 5-A, in this scenario, the no-idling games involved $v_{2}$, $v_{5}$, $v_{6}$ and $v_{10}$ that finished earlier than the rest; hence, multiple no-idling games were initiated containing different subset of these players depending on the size of $\kappa_{1}$. Clearly, it shows that with a larger $\kappa_{1}$, $ToTD$ is reduced at different target discovery percentages. Moreover, when $\kappa_{1}=3,4$, and when $\kappa_{1}=6,7$, $ToTD$ are overlapping. This is due to the same task reallocation decisions in the corresponding no-idling games.

Next, we fix $\kappa_{1}=6$, as was used in the previous scenarios, and measure $ToTD$ at different target discovery percentages when $\kappa_{2}$ varies from $2$ to $7$. Fig. LABEL:fig:vary_kappa_2 shows the results in Scenario $3$, where $v_{4}$ failed during exploration.

As defined in Section 4-B, the neighborhood size $\kappa_{2}$ equals the number of players for resilience games, i.e., $|\mathscr{P}|=\kappa_{2}$. Thus, as $v_{4}$ failed, a larger $\kappa_{2}$ could benefit the team via involving more players in the resilience game for optimization. Note that for $\kappa_{2}=7$, all live robots in the team participated in the resilience game. It is seen in Fig. LABEL:fig:vary_kappa_2 that $ToTD$ is reduced when $\kappa_{2}$ increases. Moreover, when $\kappa_{2}=2,3,4$ and when $\kappa_{2}=5,6$, the $ToTD$ are almost the same. This is because the same task reallocation decisions were made in the resilience games.

5-E Performance in the Presence of Uncertainties

In practice, uncertainties in the robot sensing systems could affect the coverage performance. Thus, for uncertainty quantification, noise was injected into the measurements of laser, compass and localization system for each robot. Typically, the uncertainty in laser measurement is $1\%$ of its sensing range, while a modestly priced compass can be as accurate as $1^{o}$ [47]. These errors were simulated with Additive White Gaussian Noise (AWGN) with standard deviations of $\sigma_{laser}=1.6cm$ and $\sigma_{compass}=0.5^{o}$, respectively. On the other hand, indoor localization systems [48] can achieve an accuracy of $0.02m$, while Real-Time Kinematic (RTK) based GPS system can be as precise as $0.05m$ [47]. Therefore, the uncertainty due to localization system is investigated using AWGN at various levels of $\sigma=0.05m$, $0.10m$ and $0.15m$. Fig. 13 shows the minimum, mean and maximum coverage ratios over five runs under different $\sigma$ for the three scenarios using $10$ robots.

5-F Computation Time for Task Reallocation

As explained earlier, once a resilience game or no-idling game is triggered, the Max-Logit algorithm was used to rapidly converge to the optimal equilibrium. This section evaluates the computation time using Algorithm 1 for different numbers of players (i.e., $|\mathscr{P}|$) and computation cycles (i.e., $L$).

As an example, Fig. 14 shows the average computation time of game $G_{1}$ in Scenario 2 under five runs. It is observed that the computation time monotonically increases as more players are involved; however, due to a distributed computation framework, the slope of growth is gentle. On the other hand, for a fixed number of players, the computation time is proportional to the number of computation cycles.

Note that if less players are involved in a resilience game, a larger number of non-player robots will be able to continue exploration during the task reallocation computation, which facilitates a smooth operation under failures; however, a game with less number of players and tasks may result in a sub-optimal reallocation decision for the whole team. Therefore, the selection of game size must consider these tradeoffs.

5-G Practical Applications of CARE

Some practical applications of CARE are listed below.

• •

Cleaning tasks: The floor cleaning task [49] in a manufacturing factory environment is one example where a team of robots could be assigned to clean up a large factory floor containing unmapped obstacles. The spread of dirt on the floor can be treated as target distribution with appropriate modifications in the formulation, and based on the day to day experience, the planner could get a good estimate of the heavy or light dirty regions daily. Under these conditions CARE can be implemented to a team of cleaning robots. Also, it is very much possible that some robot fails, thus the nearby robots can be reallocated to help it immediately if needed based on the task priorities. Other similar cleaning application examples using multiple robots include shopping malls, train stations, airports, and commercial buildings.

• •

De-mining in a hazardous environment (e.g., underwater mine countermeasures using UUVs [50]): This is an example of a non-cleaning time-critical application, where a robot team is expected to efficiently find all hidden mines even under possible failures of a few robots. In this case, the mines are the targets, and the environment is usually unknown and dangerous, thus CARE can be practically useful for efficient and resilient operation.

• •

Agriculture: There could be coverage applications in agriculture [51] for seeding and crop-cutting tasks.

6 Conclusions and Future Work

This paper presents a multi-robot coverage algorithm for resilient and efficient coverage of a priori unknown environments. The resilience and efficiency of the system are addressed via event-driven task reallocations, using game theoretic solutions. The reallocation decisions are determined by the optimal equilibrium, which is analytically shown to increase the team potential gain. Further, the efficacy of this algorithm has been validated in complex obstacle-rich scenarios on a high-fidelity robotic simulator. The results show that CARE guarantees complete coverage even in presence of failures of some robots. Also, it shows superior coverage performances as compared to three alternative methods in terms of less coverage time and faster target discovery progress.

Future research areas include: i) opportunistic scheduling [52] to enhance the speed of target discovery, ii) extension of the CARE algorithm to account for restricted communication, iii) integration of SLAM [53] with multi-robot control in the absence of localization devices, iv) consideration of threat levels in different tasks to compute the probability of success, and v) consideration of motion constraints [54][55] for the mobile robots.