Algorithms for Routing of Unmanned Aerial Vehicles with Mobile Recharging Stations

TL;DR

This paper introduces a novel algorithm for planning UAV tours with mobile and stationary recharging stations, optimizing routes and recharge points in a complex variant of the TSP, with practical solutions demonstrated through simulations.

Contribution

It formulates a new TSP variant involving mobile recharging stations and provides an algorithm that optimally plans UAV routes, recharge points, and UGV support, addressing NP-hardness.

Findings

Algorithm effectively plans UAV routes with mobile recharging stations.

Simulation results show practical running times for realistic instances.

Supports minimizing UGVs needed for uninterrupted UAV missions.

Abstract

We study the problem of planning a tour for an energy-limited Unmanned Aerial Vehicle (UAV) to visit a set of sites in the least amount of time. We envision scenarios where the UAV can be recharged along the way either by landing on stationary recharging stations or on Unmanned Ground Vehicles (UGVs) acting as mobile recharging stations. This leads to a new variant of the Traveling Salesperson Problem (TSP) with mobile recharging stations. We present an algorithm that finds not only the order in which to visit the sites but also when and where to land on the charging stations to recharge. Our algorithm plans tours for the UGVs as well as determines best locations to place stationary charging stations. While the problems we study are NP-Hard, we present a practical solution using Generalized TSP that finds the optimal solution. If the UGVs are slower, the algorithm also finds the minimum number of UGVs required to support the UAV mission such that the UAV is not required to wait for the UGV. Our simulation results show that the running time is acceptable for reasonably sized instances in practice.

Equations

$$T(j,j+1)=\{\begin{array}{l}{t}_{UAV}(x_{\sigma (j)},x_{\sigma (j+1)})\\ \max \{t_{UGV}(x_{\sigma (j)},x_{\sigma (j+1)}),r(k^{\prime }-k)\}\end{array}$$

$$T(j)=r(k^{\prime }-k).$$

$$T(\Pi )=T(1)+\sum _{j=1}^{n-1}T(j+1)+T(j,j+1)$$

$$T_{\text{I}}(x_i^k,x_j^{k^{\prime }})=t_{UAV}(x_i,x_j)$$

$$T_{\text{II}}(x_i^k,x_j^{k^{\prime }})=\max (r(k^{\prime }-k),t_{UGV}(x_i,x_j))+t_{TO}+t_L$$

$$T_{\text{III}}(x_i^k,x_j^{k^{\prime }})=t_{UAV}(x_i,x_j)+r(k^{\prime }-k+||x_i-x_j|{|}_2)+t_{TO}+t_L$$

$$T(x_i^k,x_j^{k^{\prime }})=\min \{T_{\text{I}}(x_i^k,x_j^{k^{\prime }}),T_{\text{II}}(x_i^k,x_j^{k^{\prime }}),T_{\text{III}}(x_i^k,x_j^{k^{\prime }})\}$$

$$\max \sum _{i=1}^{l-1}\sum _{j=i+1}^l{y}_{ij}$$

$$\sum _{i=1}^{j-1}{y}_{ij}\le 1\forall j,$$

$$\sum _{j=i+1}^l{y}_{ij}\le 1\forall j,$$

$$y_{ij}=0\text{ if }{T}^{{\Pi }^{\ast }}(g_i,g_j)<t_{UGV}(g_i,g_j),\text{ and}$$

$$y_{ij}=1\text{ if }{y}_{ij}\text{ is Type II edge.}$$

Full text

I Introduction

Unmanned Aerial Vehicles (UAVs) are being increasingly used for applications such as surveillance [14], package delivery [25], infrastructure inspection [11, 20], environmental monitoring [8], and precision agriculture [6, 27]. However, most small, multi-rotor UAVs have limited battery lifetime (typically $<$ 30 minutes) which prevents them from being used for long-term or large scale missions. There is significant work that is focused on extending the lifetimes of UAVs through new energy harvesting designs [17], automated battery swapping [28], low-level energy-efficient controllers [9], and low-level path planning [16]. In this paper, we investigate the complementary aspect of high-level path planning with an emphasis on energy optimization.

This work is motivated by persistent monitoring applications [21] where the UAVs are tasked with monitoring a finite set of sites on the ground by flying above these sites. The objective is to minimize the time required to visit all sites. In the absence of any additional constraints, this can be formulated as Traveling Salesperson Problem (TSP) which is a classic optimization problem [3]. However, when the sites are located far apart, the UAV may not have enough battery capacity to fly the entire tour. We consider scenarios where the UAVs are capable of landing on recharging stations and then taking off and continuing the mission. The recharging stations can either be stationary or placed on Unmanned Ground Vehicles (UGVs) which can charge the UAV while simultaneously transporting it from one site to another (Figure 1). This leads to a new variant of TSP where the output is not only a path for the UAV, but also a charging schedule that determines where and how much to recharge the UAV battery as well as paths for the UGVs. For a single UGV to keep up with the UAV, we also study the problem of minimizing the number of UGVs. Since it is not always possible for the UGV to keep up with the UAVs speed we implement a solver that can find the minimum number of UGVs necessary to service the UAV.

This problem generalizes Euclidean TSP  [3] and is consequently NP-Hard. As such, assuming P$\neq$NP, no algorithm can guarantee the optimal solution in polynomial time. Instead, we seek algorithms that find the optimal solution in reasonable time for practical instances, similar to recent works [13, 26]. Our main contribution is to show how to formulate both problems, described in section III, as Generalized TSP (GTSP) [19] instances. Earlier works have shown that a GTSP-based algorithm finds solutions faster than an Integer Programming approach [22, 13, 26]. We empirically evaluate two approaches to solve the GTSP instances: (1) GLNS solver [22] which uses heuristics to find potentially sub-optimal solutions in short time; and (2) an exact solver which reduces GTSP into TSP instances which are solved using concorde [2]. We compare the time required to find the solution in both approaches.

II Related Work

In this section we briefly describe the works related to UAV recharging stations.

II-A Recharging and Replacing UAV Batteries

A number of solutions for autonomous charging of UAVs have been proposed in the recent past. Cocchioni et al. [5] presented a vision system to align the UAV with a stationary charging station. A similar design was presented by Mulgaonkar and Kumar [18] which included magnetic contact points. There are also commercial products (e.g., the SkySense system [1]) that provide similar capabilities.

The alternative to recharging batteries is to swap them. Swieringa et al. [24] presented a “cold” swap system for exchanging the batteries for one or more helicopters. The authors evaluated their system through simulations with three helicopters where they demonstrated an increase in system lifetime from six minutes to thirty two minutes. Toksoz et al. [28] presented the design of a stationary battery swapping station for multi-rotor systems. Their design has a “dual-drum structure” that can hold a maximum of eight batteries which can be “hot” swapped.

The work presented in this paper is complementary to these hardware designs — any of the existing systems could be leveraged. Instead we show how to optimize the performance by careful placement of charging stations or planning of paths for mobile charging stations.

II-B Planning for Energy Limited UAVs

A typical strategy to deal with limited battery life of UAVs is to use multiple robots with possibly redundancy built in. Derenick et al. [7] presented a control strategy to carry out persistent coverage missions with robot teams which balances a weighted sum of mission performance and the safety of the UAVs. Mitchell et al. [15] presented an online approach for maintaining formations while substituting UAVs running low on charge with recharged UAVs. Liu and Michael [10] presented a matching algorithm for assigning UAVs with UGVs acting as recharging stations.

In our previous work [27], we showed how to plan tours for a symbiotic UAV+UGV where the UGV can mule the UAV between two deployment locations such that the UAV does not spend any energy. However, the previous work did not model the capability of UGV recharging the UAV along the way. Consequently, the goal was to maximize the number of sites that can be visited in a single charge. This results in a variant of an NP-Hard problem known as orienteering [4]. In the current work, we allow for a more general model which has the added complication of keeping track of the energy level of the UAV as well as deciding where and how much to charge along the tour.

The work by Rathinam et al. [23] looks into how to plan paths for mobile charging stations. The author of this paper studies the problem of planning UAV paths with fuel constraints and stationary refueling locations. The algorithms that the author presents allow for the planning of multiple UAVs to visit every site. This paper is slightly different than ours in the sense that the refueling stations are stationary and the UAV has to adjust to the refueling station. In our paper we allow the refueling station to mobile and adjust for the UAV allowing for shorter tour times.

The work most closely related to ours is that of Maini and Sujit [12]. They present an algorithm that plans paths for one UAV and one recharging UGV to carry out surveillance in an area. The UGV moves on a road network. The authors create an initial path for the UGV and then create a path for the UAV. In this paper, we simultaneously create paths for the UAV and UGV. Additionally, we guarantee that our algorithm finds the optimal solution for the problem.

III Problem Formulation

In this section we formally define the problem. Throughout the paper, we focus on the main problems of planning with limited battery lifetime.

The input to our algorithm is a set of $n$ sites, $x_{i}$, that must be visited by the UAV. We start with a list of common assumptions:

unit rate of discharge (1% per second); 2. 2.

UAV has an initial battery charge of 100%; 3. 3.

UAV and UGVs start at a common depot, $d$; 4. 4.

all the sites are at the same altitude; 5. 5.

UAV can fly between any two sites if it starts at 100% battery level; 6. 6.

UGVs have unlimited fuel/battery capacity.

All but the last assumption are only for the sake of convenience and ease of presentation and can be easily relaxed. Although UGVs cannot have unlimited operational time, it is a reasonable assumption since UGVs can have much larger batteries or can be refueled quickly.

We also provide a list of standard terminology that will be used throughout this paper:

• •

$x_{i}$ denotes the $i^{th}$ site that must be visited111Note that $x_{i}$ does not mean that is the $i^{th}$ point that will be visited. The order of visiting the points is determined by the algorithm. by flying to a fixed altitude;

• •

$r$ represents the time required to recharge the battery by a unit %;

• •

$t_{TO}$ is the time it takes to take off from the UGV;

• •

$t_{L}$ is the time it takes to land on the UGV;

• •

$t_{UAV}(x_{i},x_{j})$ and $t_{UGV}(x_{i},x_{j})$ give the time taken by the UAV and UGV to travel from $x_{i}$ to $x_{j}$.

Suppose $\Pi$ is a path that visits the sites in the order given by $\sigma:\{1,\ldots,n\}\rightarrow\{1,\ldots,n\}$ where $\sigma(j)=i$ implies $x_{i}$ is the $j^{th}$ point visited along $\Pi$. The cost of an edge from $x_{\sigma(j)}$ to $x_{\sigma(j+1)}$ along $\Pi$ depends on whether the UAV flies between the two sites or if it is muled by the UGV between the two sites while being recharged. Let $k$ and $k^{\prime}$ be the battery levels at $\sigma(j)$ and $\sigma(j+1)$. Therefore,

[TABLE]

In addition, we also have non-zero node costs if the UAV is charged from battery level $k$ to $k^{\prime}$ at a site $x_{i}$ rather than along an edge:

[TABLE]

Therefore, the total path cost is given by,

[TABLE]

We are now ready to define the problems studied in this paper.

Problem 1** (Multiple Stationary Charging Stations (MSCS)).**

Given a set of sites, $x_{i}$, to be visited by the UAV, find a path $\Pi^{*}$ for the UAV that visits all the sites as well as select one or more sites (if needed) to place recharging stations so as to minimize the total time given by Equation 3 under the assumptions given above.

Problem 2** (Single Mobile Charging Station (SMCS)).**

Given a set of sites, $x_{i}$, to be visited by the UAV, find a path $\Pi^{*}$ for the UAV that visits all the sites as well as another path for the UGV acting as a mobile basestation so as to minimize the total time given by Equation 3 under the assumptions given above. Assume that the UAV and UGV travel at the same speed.

The assumption that the UGV is as fast as the UAV is not necessary to find a solution; it is required to guarantee optimality for one UGV. If the UGV is slower than the UAV, we can still use the paths returned by the algorithm for one UGV, but the UAV may have to wait. An alternative is to minimize the number of UGVs required to ensure the UAV never has to wait for a recharging station.

Problem 3** (Multiple Mobile Charging Stations (MMCS)).**

Given a path, $\Pi^{*}$, for a UAV and a set of charging sites as well as Type II edges for the UGVs, find the minimum number of slower UGVs necessary to service the UAV, without the UAV having to wait for a UGV. The input to MMCS is obtained by solving SMCS, without assuming the single UGV in SMCS is as fast as the UAV.

Our main contribution is a GTSP-based algorithm that solves the first two problems optimally and an Integer Linear Programming (ILP)-based algorithm that solves Problem 3 optimally. As mentioned previously, Problems 1 and 2 are NP-Hard and consequently finding optimal algorithms with running time polynomial in $n$ is infeasible under standard assumptions. Instead, we provide a practical solution that is able to solve the three problems to optimality in reasonable time (quantified in Section V).

IV GTSP-Based Algorithm

In this section we show how to formulate Problems 1 and 2 as GTSP instances [19]. The input to GTSP is a graph where the vertices are partitioned into clusters. The objective is to find a minimum cost tour that visits exactly one vertex per cluster. When each cluster contains only one vertex, the GTSP reduces to TSP.

Solving GTSP is at least as hard as solving TSP. However, Noon and Bean [19] presented a technique to convert any GTSP input instance into an equivalent TSP instance on a modified graph such that finding the optimal TSP tour in the modified graph yields the optimal GTSP tour in the original graph. We can solve GTSP by solving TSP optimally using a numerical solver and we use concorde [2], which is the state-of-the-art TSP solver or GLNS [22], that is a heuristics-based GTSP solver. The results in Section V show that GLNS significantly faster than concorde. However, only the concorde approach is guaranteed to find the optimal solution.

We start by showing how to formulate the SMCS and MSCS problems as GTSP instances. After obtaining an output, we can convert the TSP solution back into a GTSP solution, then into a solution for the SMCS or MSCS problems. The process of converting SMCS and MSCS into TSP is the same. Only the process of converting the solution of TSP to solutions of SMCS and MSCS differ.

IV-A Transforming SMCS/MSCS to GTSP

Given an SMCS or MSCS instance, we show how to create a GTSP instance consisting of a directed graph where the vertices are partitioned into non-overlapping clusters. We create one cluster, $g_{i}$, for each input site $x_{i}$. Each cluster, $g_{i}$ has $m$ vertices, each one corresponding to a discretized battery level. That is, $g_{i}=\{x_{i}^{k}\mid\forall i\in[1:n],\forall k\in\{1,2,\ldots,m\}\}$. $x_{i}^{k}$ represents the UAV reaching site $x_{i}$ with $k*\frac{100\%}{m}$ battery remaining. $m$ is an input discretization parameter. Figure 2 shows the six clusters for six input sites with $m=5$.

Next we describe how to create the edges amongst the vertices in the $n$ clusters. We create three types of edges. Type I edge between $x_{i}^{k}$ and $x_{j}^{k^{\prime}}$ models the case where the UAV directly flies from $x_{i}$ to $x_{j}$. The cost of a Type I edge is given by:

[TABLE]

A Type I edge exists between $x_{i}^{k}$ and $x_{j}^{k^{\prime}}$ if and only if $k-k^{\prime}$ equals the distance between $x_{i}$ and $x_{j}$. For ease of exposition, we assume that taking-off and landing energy consumption is negligible. Nevertheless, we can easily incorporate this in the edge definitions. These types of edges are shown by the red lines in Figure 2.

A Type II edge from $x_{i}^{k}$ to $x_{j}^{k^{\prime}}$ models the UAV landing on the UGV at $x_{i}$ and recharging while being muled to $x_{j}$ by the UGV. The cost of a Type II edge is given by:

[TABLE]

The cost is the maximum of the time taken to recharge from $k$ to $k^{\prime}$ and the time it takes the UGV to travel from $x_{i}$ to $x_{j}$. Note that a Type II edge exists only if $k^{\prime}\geq k$. Type II edges are shown as the green edges in Figure 2.

Finally, we have Type III edges that represent the UAV flying from $x_{i}$ to $x_{j}$ and then landing on the UGV at $x_{j}$ and recharging up to $k^{\prime}$ battery level. The cost of a Type III edge is given by:

[TABLE]

A Type III edge exists if and only if $k^{\prime}\geq k-||x_{i}-x_{j}||_{2}$. Figure 2 shows the Type III edges in blue.

Only Type I and Type III edges exist when solving MSCS whereas all three edges are possible when solving MMCS. Note that Type II and Type III edges require the UAV to take off and land at every site. This prevents the UAV from not taking off between two consecutive Type II edges. This is because in order to visit a site it must fly to a fixed altitude to consider the site visited.

There are certain pairs of vertices for which more than one type of edge may be allowed. In such a case, we pick the minimum of the three edge costs (assuming the edge cost is $\infty$ if the edge does not exist) and assign the minimum cost for the edge. That is, the edge cost $T(x_{i}^{k},x_{j}^{k^{\prime}})$ is given by:

[TABLE]

We also create an $n+1^{th}$ cluster containing a dummy vertex called as the depot, $d$. We add a zero cost edge from $d$ to all vertices, $x_{i}^{k}$, with $k=m$ and edges from all vertices back to $d$. The reason to create a depot node is that the TSP solver finds a closed tour whereas we are interested in finding paths.222A path visits a vertex exactly once whereas a tour has the same starting and ending vertices. The depot node serves to ensure that we can find a closed tour without charging for the extra edges.

IV-B Converting Optimal TSP Tour to UAV and UGV Paths

An optimal TSP tour immediately yields an optimal GTSP solution. The order in which the clusters are visited gives the sequence of vertices on the UAV paths. What remains is deciding the UGV path for SMCS and recharging station placements for MSCS.

In MSCS, we only have Type I and Type III edges. If a Type III edge, say from $x_{i}^{k}$ to $x_{j}^{k^{\prime}}$, appears in the GTSP solution, then we will place a recharging station at the site $x_{j}$. No recharging stations are placed for Type I edges in the solution.

In MMCS, all three edges are possible, whereas only Type I and Type II in SMCS. We check the type of each edge in the GTSP solution, one by one. If a Type I edge appears in the GTSP solution, then it does not affect the UGV tour. If a Type II edge, say from $x_{i}^{k}$ to $x_{j}^{k^{\prime}}$, appears in the GTSP solution, we add $x_{i}$ and $x_{j}$ to the UGV path (in this order). If a Type III edge, say from $x_{i}^{k}$ to $x_{j}^{k^{\prime}}$, appears in the GTSP solution we add only $x_{j}$ to the UGV path. The UGV path, as a result, visits a subset of the input sites. If the UGV is slower than the UAV, then it is possible that the UAV will reach a site before the UGV does and will be forced to wait. We implement an ILP that allows us to solve for the minimum number of UGVs necessary to service the UAV without waiting (shown in section IV-C).

Theorem 1**.**

The GTSP-Based algorithm finds the optimal solution for SMCS and MSCS assuming that there exists an optimal TSP solution.

The proof follows directly from the proof of optimality of the GTSP reduction given by Noon and Bean [19].

IV-C Solution for Problem 3

We present a solution to the MMCS problem based on an ILP formulation. The input is obtained by solving Problem 2, where we are given a UAV path and a set of UGV sites and Type II edges. The UGV path visits only a subset of the sites in $\{x_{i}\}$. We denote these sites by $\{g_{1},g_{2},\ldots,g_{l}\}$, where $l\leq n$. For each edge from $g_{i}$ to $g_{j}$, where $(j>i)$, we associate the variable $y_{ij}$, which equals 1 if the edge will be traversed by some UGV and 0 otherwise. Also associated with each edge is the time to come for the UAV and UGV denoted by $T^{\Pi^{*}}(g_{i},g_{j})$ and $t_{UGV}(g_{i},g_{j})$ respectively. Here $T^{\Pi^{*}}(g_{i},g_{j})$ is the time taken by the UAV to fly the subpath of $\Pi^{*}$ from $g_{i}$ to $g_{j}$, which may contain some intermediate sites. $t_{UGV}(g_{i},g_{j})$, on the other hand is the time for the UGV to directly go from $g_{i}$ to $g_{j}$. Using the above notation we provide the ILP formulation given as follows:

[TABLE]

subject to:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Equation 5 and 6 only allow a maximum of one incoming edge and a maximum of one outgoing edge. The constraint given by Equation 7 removes all UGV edges where the UAV would have to wait for the UGV. Lastly Equation 8 forces our problem to use Type II edges if present. Using the above equations we are able to solve Problem 3.

V Evaluations

In this section, we present simulation and preliminary experimental results using the proposed algorithm.

V-A Effect of the Parameters

Figure 3 shows the outputs obtained for different configurations of the $t_{TO},t_{L},r,t_{UGV}$ parameters for the same 20 input sites and with $m=10$ battery levels. Each figure has the UAV+UGV tour with blue solid edges (only UAV), red solid edges (only UGV), green solid edges (UAV and UGV separate), and red dashed edges (UAV+UGV together).

We make the following intuitive observations using the six cases shown in Figure 3:

• •

$t_{TO}=0$, $t_{L}=0$ and $r=0$: UAV does not differentiate between the type of the edge because there is no penalty to recharge (Figure 3(a));

• •

$t_{TO}+t_{L}>0$: recharging has a penalty and as such the number of recharging stops are reduced (Figures 3(d), 3(e) and 3(f));

• •

$t_{TO}=0$, $t_{L}=0$, $r=0$ and $t_{UGV}>t_{UAV}$: the UAV will use Type III edges for charging because $t_{UGV}$ will make Type II edges higher cost (Figure 3(b) and 3(e));

• •

$t_{TO}=0$, $t_{L}=0$, $r>0$ and $t_{UGV}=t_{UAV}$: the UAV will use Type II edges for charging instead of Type III edges (Figure 3(c));

We observe that the recharge time $r$ and UGV speed $t_{UGV}$ affect which type of edges are used. If the time it takes to recharge is much larger than $t_{UGV}$ then the UAV will favor Type II edges and when the time it takes to recharge is much less than $t_{UGV}$ then the UAV will favor Type III edges.

V-B Computational Time

We use two solvers for the MSCS and SMCS problems. When using concorde we obtain an optimal solution, but with more computational time. Therefore, it can only solve smaller instances. The GLNS solver can solve larger instances, but cannot always guarantee optimality. Nevertheless, we observe that the GLNS was able to find the optimal solution for all of the cases reported in Figure 4.

Figure 4 shows a direct comparison of the computational times of the two methods. We compared the two methods by first varying the amount of inputs sites (Figure 4(a)), with $m=4$, and then comparing the two by varying the amount of battery levels (Figure 4(b)), given $n=12$. We ran 10 trials with random input sites. We plot the average value along with the maximum and minimum value.

Due to limitations in concorde, we were not able to run larger instances, but GLNS can run larger instances, as shown in Figure 5. We show the effect of incrementing $n$ from 20 to 50 and $m$ from 50 to 150 in Figure 5. We ran 5 random instances and plot the average of the 5 instances with the maximum and minimum value.

Figure 6 shows the minimum number of UGVs necessary to service a single UAV. We used the same $m=50$ data set that was used to create Figure 5. The plot shows the effects of having a slower UGV.

V-C Preliminary Field Experiments

We also carried out preliminary field experiments using the quadrotor and Husky UGV (Figure 1). The input consisted of 50 sites shown in Figure 7(a) and $m=100$ battery levels. We restricted the size of the area to a $200\times 100$ meters area and set $t_{TO}=4$, $t_{L}=4$, $r=0$, and $T_{UGV}=T_{UAV}$. Figure 7 shows the results of the experiments. The outputs for the UAV tour and UGV tour are shown in Figure 7(a). Figure 7(b) shows the output data from the Pixhawk flight controller. The UAV was programmed to fly autonomous GPS missions. The autonomous missions were executed by sending a set of waypoints that created a path for the UAV to travel and meet up with the UGV. Once at the final waypoint on the path, the we manually piloted the UAV to land on the UGV. The UGV was manually driven to the next take-off point in the tour. This next waypoint could be a new waypoint due to Type II edge or the same waypoint due to Type III edge. The UAV then detects if it is at the next take-off point by comparing its current GPS position with the GPS coordinates of the next take-off point. This process loops until the UAV reaches its final waypoint whereupon the UAV would autonomously land on the ground. Additional experiments can be seen in the multimedia submission.

VI Conclusion and Future work

In this paper, we present an optimal algorithm for routing a battery-limited UAV and a mobile recharging station to visit a set of sites of interest. We are also conducting larger scale experiments using a recharging station being developed in-house. The longer-term future work is to design algorithms to handle multiple UAVs and UGVs as well as stochastic energy consumption models.