Motion Planning for Multi-Mobile-Manipulator Payload Transport Systems

TL;DR

This paper introduces a hierarchical, real-time motion planning algorithm for multi-mobile-manipulator systems to enable collision-free payload transport, validated through simulations in dynamic environments.

Contribution

A novel hierarchical approach combining virtual bounding boxes and decentralized model-predictive control for scalable, real-time multi-robot payload manipulation.

Findings

Successfully plans collision-free trajectories in simulations

Scalable to multiple robots in dynamic environments

Real-time performance demonstrated

Abstract

In this paper, a kinematic motion planning algorithm for cooperative spatial payload manipulation is presented. A hierarchical approach is introduced to compute real-time collision-free motion plans for a formation of mobile manipulator robots. Initially, collision-free configurations of a deformable 2-D virtual bounding box are identified, over a planning horizon, to define a convex workspace for the entire system. Then, 3-D payload configurations whose projections lie within the defined convex workspace are computed. Finally, a convex decentralized model-predictive controller is formulated to plan collision-free trajectories for the formation of mobile manipulators. This approach facilitates real-time motion planning for the system and is scalable in the number of robots. The algorithm is validated in simulated dynamic environments. Simulation video: https://youtu.be/9EKj7RwRs_4.

Figures

Equations

$$do{f}_s=\lambda (l-1-j)+\sum _{j}do{f}_i,$$

$$do{f}_s=6((5K+2)-1-6K)+6K,$$

$$6((l_{k}K+2)-1-j_{k}K)+K(do{f}_k)$$

$$6(l_k-j_k)+do{f}_k$$

$${\mathbf{x}}_t^{B_d}={\mathbf{x}}_t^T-\left[d_{des}cos({\psi }_t^{T,B})d_{des}sin({\psi }_t^{T,B})\right],$$

$$J_{\mathrm{MPC}}=\sum _{n=0}^H({\mathbf{u}}_t^B(n)\text{boldmath}{\Omega }_u{\mathbf{u}}_t^B(n{)}^{\top }+$$

$$({\mathbf{x}}_t^B(n+1)-{\mathbf{x}}_t^{B_d})\text{boldmath}{\Omega }_x({\mathbf{x}}_t^B(n+1)-{\mathbf{x}}_t^{B_d}{)}^{\top }),$$

$$F^{i}(d)=\begin{cases}F_{max}\ &\text{if}~{}~{}d<d_{min}\\ \frac{\pi}{2}\big{(}\frac{cot(z)+z-\frac{\pi}{2}}{d_{max}-d_{min}}\big{)}&\text{if}~{}~{}d_{min}\leq d\leq d_{max}\\ 0,&\text{if}~{}~{}d>d_{max}\end{cases}\;.$$

$${\mathbf{f}}_{dyn}(n)=\sum _i^N{F}^{D_i}(d_{dyn}(n)){\beta }^{D_i}(n),$$

$${\mathbf{f}}_{sta}(n)=\sum _i^M{F}^{S_i}(d_{sta}(n)){\beta }^{S_i}(n),$$

$${\mathbf{f}}_{rep}^B(n)={\mathbf{f}}_{dyn}(n)+{\mathbf{f}}_{sta}(n)+{\text{f}}_{ang}(n)$$

$${\mathbf{f}}_t^B(n)=\{\begin{array}{ll}{\mathbf{f}}_{rep}^B(n)& \text{if}\Vert {\mathbf{f}}_{rep}^B(n)\Vert <F_{max}\\ F_{max}\frac{{\mathbf{f}}_{rep}^B(n)}{\Vert {\mathbf{f}}_{rep}^B(n)\Vert }& \text{if}\Vert {\mathbf{f}}_{rep}^B(n)\Vert \ge F_{max}\end{array}$$

$$w_t^B(n+1)$$

$$r_t^B(n+1)$$

$${\mathbf{x}}_t^{B\ast }(1)\dots {\mathbf{x}}_t^{B\ast }(H+1),{\mathbf{u}}_t^{B\ast }(0)\dots {\mathbf{u}}_t^{B\ast }(H)=\underset{{\mathbf{u}}_t^B(0)\dots {\mathbf{u}}_t^B(H)}{\mathrm{argmin}}(J_{MPC})$$

$${\mathbf{x}}_t^B(n+1{)}^{\top }=\mathbf{A}{\mathbf{x}}_t^B(n{)}^{\top }+\mathbf{B}({\mathbf{u}}_t^B(n)+{\mathbf{f}}_t^B(n){)}^{\top },$$

$${\mathbf{u}}_{\min }^B\le {\mathbf{u}}_t^B(n)\le {\mathbf{u}}_{\max }^B,$$

$${\mathbf{x}}_{\min }^B\le {\mathbf{x}}_t^B(n+1)\le {\mathbf{x}}_{\max }^B.$$

$${\varphi }_t^{\ast }(n),{\omega }_t^{\ast }(n)=\underset{{\omega }_t(0)\dots {\omega }_t(H_p)}{\mathrm{argmin}}\sum _{n=0}^{H_p}{\text{w}}_1{\varphi }_t(n+1{)}^2+{\text{w}}_2{\omega }_t(n{)}^2$$

$${\mathbf{Reg}}_t^B(n+1){\hat{T}}_P^F({\varphi }_t(n+1),{\mathbf{x}}_t^P(n+1),{\mathbf{v}}_i{)}^{\top }\le {\mathbf{b}}_t^B(n+1),\forall i,$$

$${\varphi }_t(n+1)={\varphi }_t(n)+{\omega }_t(n)\Delta t,$$

$${\varphi }_{min}\le {\varphi }_t(n+1)\le {\varphi }_{max},$$

$${\omega }_{min}\le {\omega }_t(n)\le {\omega }_{max},$$

$$J_{DMPC}=\sum _{n=0}^{H_r}{\mathbf{u}}_t^k(n)\text{boldmath}{\Omega }_{\mathbf{u}}{\mathbf{u}}_t^k(n{)}^{\top }+$$

$$({\mathbf{x}}_t^k(n+1)-{\mathbf{x}}_{ref}^k(n+1))\text{boldmath}{\Omega }_{\mathbf{x}}({\mathbf{x}}_t^k(n+1)-{\mathbf{x}}_{ref}^k(n+1){)}^{\top }$$

$${\mathbf{x}}_t^{k\ast }(1)\dots {\mathbf{x}}_t^{k\ast }(H_r+1),{\mathbf{u}}_t^{k\ast }(0)\dots {\mathbf{u}}_t^{k\ast }(H_r)=\underset{{\mathbf{u}}_t^k(0)\dots {\mathbf{u}}_t^k(H_r)}{\mathrm{argmin}}{J}_{DMPC}$$

$${\mathbf{x}}_t^k(n+1{)}^{\top }=\mathbf{A}{\mathbf{x}}_t^k(n{)}^{\top }+\mathbf{B}({\mathbf{u}}_t^k(n)+{\mathbf{f}}_t^k{)}^{\top }$$

$${\mathbf{Reg}}_t^k(n+1){\mathbf{x}}_t^k(n+1{)}^{\top }\le {\mathbf{b}}_t^k(n+1)$$

$${\mathbf{u}}_{\min }^k\le {\mathbf{u}}_t^k(n)\le {\mathbf{u}}_{\max },$$

$${\mathbf{x}}_{\min }^k\le {\mathbf{x}}_t^k(n+1)\le {\mathbf{x}}_{\max }^k.$$

Full text

I Introduction

Coordination between robotic agents to collectively perform payload transportation tasks has recently piqued the interest of the robotics community [1, 2, 3]. In this paper, we address the problem of local motion planning for cooperative payload manipulation in dynamic environments. Each robot in the system is associated with a 6-degrees-of-freedom (6-dof) manipulator and a holonomic mobile base. Each manipulator grasps a common payload as shown in Fig. 1. The resulting system is a mobile parallel manipulator which is capable of non-planar payload manipulation. Planning real-time motion for such a setup in dynamic environments is challenging because of, (i) the high dimensional system configuration space, (ii) real-time environmental obstacles, and, (iii) inter-agent collision avoidance constraints which restrict the system to operate in a non-convex workspace.

In this work, we present three key contributions to address the above mentioned challenges. (1) We propose a hierarchical motion planning algorithm which enables real-time non-planar mobile manipulation of a common payload in dynamic environments. (2) As part of this motion planning, we introduce a novel model-predictive controller (MPC) to identify real-time collision-free configurations of a deformable virtual bounding box. The planned bounding box configurations form a multi-scale convex workspace for the robots and the payload over the planning horizon. Subsequently, we identify feasible configurations of the payload which lie within the planned deformable virtual bounding boxes to guarantee environmental collision avoidance. (3) We then formulate a novel formation controller based on decentralized MPC to generate kinematically feasible, collision-free, trajectories for each robot. The manipulator configuration associated with each robot is determined using inverse kinematics between the payload grasp point and the position of the mobile base.

We validate the efficacy of our hierarchical motion planner by simulating the entire mobile parallel manipulator setup in different environments consisting of both static and dynamic obstacles. The robots are tasked to safely manipulate and navigate the payload while tracking a moving target.

II Related Work

In this section, the works pertinent to multi-robot payload transport systems are reviewed in detail.

Formation Frameworks: Multi-robot payload transportation constraints mobile robots to navigate in a formation. Leader-follower [4], behavior-based [5], virtual leader-follower [6] and virtual structure [7] are some of the formation frameworks that have been proposed in literature. We employ the virtual leader-follower framework in our work to derive decentralized motion plans.

*System Modeling and Control: * Virtual linkage models [8] were used to identify multi-grasp forces for cooperative manipulation and control. Flexible object transportation with static collision avoidance using non-holonomic mobile manipulators was explored in [9]. In [10], a screw theoretic framework was developed for planar payload mobile manipulation using multiple robots mounted with passive 2-dof manipulators. Adaptive controllers [11] were studied for dealing with unknown payload inertia parameters, external disturbances and tracking a known trajectory using multiple robots. The above works do not consider non-planar rigid payload manipulation coupled with spatiotemporal motion planning constraints.

*Constrained Optimization: * In [6], differential geometry based formation optimization was used to compute energy-efficient robot trajectories. Semi-definite [12] programming was leveraged to plan motion for robot formations, with constraints on shape templates, network-connectivity, and obstacle avoidance. Trajectory optimization [13] with linearized dynamics was solved as a sequential convex program to assign tasks and plan motion of robot swarms. Offline trajectory optimization was established for planar payload transportation using multiple passive 1-dof non-holonomic mobile manipulators [14]. An integer program [3] was proposed to geometrically plan motion for multi-mobile manipulator, non-planar payload transportation. In [15, 1] multi-robot planar mobile manipulation of a payload was achieved using non-linear optimization , which is solved as a sequential convex program. An extension to [15] was probed to holonomically manipulate deformable payloads [16].

In contrast to the above works, each mobile manipulator in our system has the freedom of having significant relative motion between its end-effector grasp point and the mobile base. This freedom grants us the ability to plan kinematically feasible and energy optimal robot trajectories. We formulate a scalable motion planning framework which enables a formation of robots to navigate through dynamic environments while manipulating a payload in 3-D space.

Sec. III briefs on the mathematical notations and the hierarchical motion planning algorithm, Sec. IV analyzes the mobile parallel manipulator, Sec. V discusses the proposed hierarchical motion planning approach in detail, Sec. VI elaborates on simulation results. We conclude in Sec. VII and discuss future work directions.

III Overview

III-A Preliminaries

Let there be $K$ mobile manipulator robots transporting a payload $P$ while tracking a moving target $T$. The position of $T$ in the world frame at time $t$ is denoted by $\mathbf{x}_{t}^{T}\in\mathbb{R}^{2}$. Typically the moving target is a person guiding the system of robots globally. The configuration $\xi_{t}^{k}$ of each robot $k\in[1\dots K]$ is defined by the pose (position $\mathbf{x}_{t}^{k}$ and yaw $\psi_{t}^{k}$) of the mobile base and the manipulator joint angles $\theta_{t}^{i,k},\forall i\in[1,6]$, $\xi_{t}^{k}=[\mathbf{x}_{t}^{k}~{}\psi_{t}^{k}~{}\theta_{t}^{1,k}..\theta_{t}^{6,k}]\in\mathbb{R}^{9}$. The position of the manipulator joints in the task space are denoted as $\mathbf{m}_{t}^{i,k}\in\mathbb{R}^{3},~{}\forall i\in[1,6]$. Each robot $k$ grasps $P$ rigidly at $\mathbf{g}_{t}^{k}\in\mathbb{R}^{3}$ $(=\mathbf{m}_{t}^{6,k})$, which is pre-defined and lies on the surface of $P$. The system of robots is enclosed by a 2-D virtual bounding-box $B$ whose configuration is denoted by $\xi_{t}^{B}=[\mathbf{x}_{t}^{B},~{}\psi_{t}^{B},~{}r_{t}^{B}]$ where $\mathbf{x}_{t}^{B}$ denotes the 2D position of its centroid, $\psi_{t}^{B}$ its yaw and $r_{t}^{B}$ the length of its half-diagonal (representing scale). The payload’s geometric center $\mathbf{x}_{t}^{P}\in\mathbb{R}^{3}$ is at a constant height offset $h^{P}$ above $\mathbf{x}_{t}^{B}$. The yaw $\psi_{t}^{P}$ is always aligned with $\psi_{t}^{B}$, s.t. $\psi_{t}^{P}\cong\psi_{t}^{B}$. Fig. 2 provides an overview of the notations described above on an illustration of a four robot multi-mobile manipulator system. Regional constraints of each mobile manipulator are also visualized in Fig. 2(a), definitions of which are further discussed in Sec. V-C.

III-B Motion Planning Overview

The operational environment consists of $M$ static obstacles and $N$ non-cooperative dynamic obstacles $(S_{1},..S_{M},D_{1},..D_{N})$. The objective of the system of robots is to ensure that the centroid of the system bounding-box $\mathbf{x}_{t}^{B}$ reaches a desired destination position $\mathbf{x}_{t}^{B_{d}}$ in the vicinity of the target position $\mathbf{x}_{t}^{T}$. Our work is motivated by the application of simultaneous target tracking ([17, 18]) and payload transportation. The key requirements in our target tracking scenario are, (i) to not lose track of the target, and, (ii) to ensure that the payload and the formation of robots avoid all the obstacles in their vicinity. To address both these objectives in an integrated approach, we formulate the hierarchical motion planning algorithm, as detailed in Fig. 3. The main steps are,

compute $\mathbf{x}_{t}^{B_{d}}$ in the direction of the target position, 2. 2.

avoid environmental obstacles with the deformable virtual bounding box using a novel MPC (section V-A) for navigation and deformation, 3. 3.

rotate the payload to ensure that its projection lies within the area occupied by $B$ over the horizon (section V-B), 4. 4.

solve a novel decentralized MPC (section V-C) to compute efficient and collision-free mobile base trajectories, 5. 5.

identify manipulator configurations based on known mobile base positions $\mathbf{x}_{t}^{k}$ and the derived end-effector grasp positions $\mathbf{g}_{t}^{k}$ from the payload configuration.

IV System Description: Mobile Parallel Manipulator

Each robot in the system consists of a holonomic 3-dof mobile base and a 6-dof manipulator. Based on the definition of $\xi_{t}^{k}$, each robot has a configuration space $\mathtt{C}^{robot}\in\mathbb{R}^{9}$. Since there are $K$ robots in the system, $\mathtt{C}^{system}\in\mathbb{R}^{9\times K}$. The system motion is planned for a fixed prediction horizon $H$ to compute trajectories for ensuring inter-robot and environmental collision avoidance (M static obstacles and N dynamic obstacles). Therefore, at every time instant the system has to identify configurations in $\mathtt{C}^{plan}\in\mathbb{R}^{9\times K\times H}$, while also being subjected to non-convex collision avoidance and kinematic constraints. For real-time motion planning, this is computationally expensive and is not scalable in the number of robots. We, therefore, decentralize the computation of robot configurations. This restricts the search space to $\mathbb{R}^{9\times H}$, albeit by increasing the communication complexity. If both the mobile base control inputs and the angular velocity of the payload (non-planar) orientation are minimized, there is an implicit energy minimization of the manipulator’s joint velocities. Therefore, we hierarchically optimize the payload orientation, and subsequently, the position of each mobile base in $\mathtt{C}^{traj}\in\mathbb{R}^{3\times H}(\subset\mathtt{C}^{plan})$ to obtain sub-optimal but feasible systemic configurations.

Remark 1 - We observe that, by using any number ($K$) of revolute 6-dof manipulators (in non-singular configurations) to rigidly grasp and manipulate a common payload, the system always has at least 6-dof. This key observation enables us to hierarchically plan the motion of the payload, mobile bases and the manipulators. We analyze this remark and its consequences below.

Analysing our system of robots as a static parallel manipulator which is rigidly affixed to the ground, the dof of the constrained system in accordance with Grübler’s formula is,

[TABLE]

where, $\lambda=6$ (for spatial systems), $l$ is the total number of links, including all the fixed links, $j$ is the total number of joints in the system, and $dof_{i}$ and $dof_{s}$ denote the degrees of freedom of each joint and the system respectively. In our spatial system, each manipulator $k$ has five links and six revolute joints $(dof_{k}=6)$. Treating the robot bases and ground plane as a single link and the payload as another link, Eq. (1) for the static parallel manipulator is computed as,

[TABLE]

where, $K$ is the number of manipulators grasping the payload. Notice that the dof of the system ($dof_{s}$) is six for any $K$. This result has the following important consequences:

(a) Scalability: The mobility of the parallel manipulator is not affected by the number of robots used. Therefore, the system is kinematically scalable.

(b) Payload Mobility: Since the parallel manipulator has 6-dof, the payload has complete mobility in 3-D space. This is limited only by the manipulator link lengths.

(c) Robot Mobility: As $dof_{s}=dof_{k}$, there exists a natural partition in the task space of the bases and manipulators. Mobile bases (3-dof) extend the degrees of freedom of each manipulator. This mobility introduces significant relative motion for the bases w.r.t. to the manipulator grasp points. This is limited only by manipulator link lengths and spatiotemporal constraints (e.g., obstacle avoidance).

(d) Decentralization: Each robot can independently compute feasible and efficient motion plans for itself.

Remark 2 - *For the manipulated common payload to have at least 6-dof mobility, each manipulator $k\in[1\dots K]$ should have at least 6-dof. This remark is conditioned on each joint being revolute and the manipulator being in a non-singular configuration.

*If each manipulator $k$ has $j_{k}$ revolute joints, $l_{k}$ links and $dof_{k}$ degree-of-freedom . For a system with $K$ robots to have atleast 6-dof, Eq. (1) is evaluated as,

[TABLE]

In our case $(l_{k}-j_{k})=-1$, which implies $dof_{k}\geq 6$. Therefore each manipulator should have at least 6 revolute joints to ensure that the payload has 6-dof.

V Hierarchical Motion Planning

In this section, we describe our proposed hierarchical motion planner in detail. Sec. V-A describes a convex optimization formulation to plan the motion of a deformable virtual bounding box (DVB) through static and dynamic obstacles. High-level motion guidance for the DVB is achieved by tracking a desired moving target. Obstacle avoidance is enforced by embedding artificial repulsive potential fields into the optimization, which aid in deforming and navigating the DVB. Sec. V-B presents an optimization program to identify non-planar configurations of the payload so as to ensure that its projections lie within the convex workspace defined by DVB. In Sec. V-C feasible, collision-free robot trajectories are computed using a convex decentralized MPC for the mobile bases.

V-A Deformable Virtual Bounding Box

To identify collision-free goal-directed motion plans for the DVB, we formulate an MPC. The objective of the MPC is to ensure that the centroid $\mathbf{x}_{t}^{B}$ reaches a desired position $\mathbf{x}_{t}^{B_{d}}$ efficiently. The MPC is constrained by obstacle avoidance, linear translational dynamics, position limits, and control saturation bounds.

V-A1 Objective

A desired position $\mathbf{x}_{t}^{B_{d}}$ of the bounding box in the vicinity of the target $\mathbf{x}_{t}^{T}$ is defined as,

[TABLE]

where, $\psi_{t}^{T,B}$ is the angle of $x_{t}^{B}$ about the target position $x_{t}^{T}$ and $d_{des}$ is a desired distance to the target. The cost of the MPC is given as,

[TABLE]

where, $\mathbf{u}_{t}^{B}(n)\in\mathbb{R}^{2}$ is the linear translational velocity control input at discrete horizon step $n$. $\boldmath{\Omega}_{u},\boldmath{\Omega}_{x}$ are diagonal positive semi-definite weight matrices for the control and state costs respectively. Eq. (V-A1) minimizes the control input and the distance between $\mathbf{x}_{t}^{B}$ and $\mathbf{x}_{t}^{B_{d}}$ over a fixed time horizon $H$.

V-A2 Environmental Obstacle Avoidance

We compute artificial repulsive potential field vectors for a planning horizon and embed them as external control inputs in the MPC dynamics constraint. This operation preserves the convexity of optimization. Incorporating external control inputs to avoid collisions within an MPC framework was presented in our previous work [19]. The repulsive potential field magnitude w.r.t the $i^{th}$ obstacle, is given as,

[TABLE]

Here, z=\frac{\pi}{2}\big{(}\frac{d-d_{min}}{d_{max}-d_{min}}\big{)}, argument $d$ is a distance metric between $\mathbf{x}_{t}^{B}$ and obstacle $i$. Note that, $F^{i}(d)$ varies hyperbolically w.r.t $d$. $d_{max}$ and $d_{min}$ are distances defining the region of influence of the potential field and the distance at which the potential field value tends to infinity respectively. In practice, the potential field at $d_{min}$ is clamped to a positive value $F_{max}\geq max(\|\mathbf{u}_{t}^{B}\|)$ to ensure obstacle avoidance.

Dynamic Obstacle Avoidance: The external control input $\mathbf{f}_{dyn}(n)$ due to the influence of $N$ dynamic obstacles is,

[TABLE]

where, argument $d_{dyn}(n)=\|\mathbf{x}_{t}^{B}(n)-\mathbf{x}_{t}^{D_{i}}(n)\|_{2},\forall n\in[1,H]$, is computed using both the prediction horizon motion plan for the bounding box and the dynamic obstacle $D_{i}$. The external control input acts along the direction $\mathbf{\beta}^{D_{i}}(n)=\frac{\mathbf{x}_{t}^{B}(n)-\mathbf{x}_{t}^{D_{i}}(n)}{\|\mathbf{x}_{t}^{B}(n)-\mathbf{x}_{t}^{D_{i}}(n)\|_{2}}$, which is a unit vector pointing in the direction away from $D_{i}$’s horizon motion plan111The state evolution model and velocity of $D_{i}$ are assumed to be known.. $d_{min}$ is the length of the half-diagonal $r_{t}^{B}(n)$ of B (see Fig. 4) and $d_{max}=d_{min}+\delta,\delta>0$.

Static Obstacle Avoidance: The total external control input due to the $M$ static obstacles is given by,

[TABLE]

where, $d_{sta}(n)=\|\mathbf{x}_{t}^{B}(n)-\mathbf{x}_{t}^{S_{i}}\|_{2}$ and $\mathbf{\beta}^{S_{i}}(n)=\frac{\mathbf{x}_{t}^{B}(n)-\mathbf{x}_{t}^{S_{i}}}{\|\mathbf{x}_{t}^{B}(n)-\mathbf{x}_{t}^{S_{i}}\|_{2}}$ . The tracked target is also considered as a static obstacle.

V-A3 Handling Field Local Minima

In cluttered static environments the DVB could stagnate in space due to field local minima. We propose the following approaches to resolve this.

Approach Angle Force: Repulsive potential fields are used to deviate the DVB away from obstacles which lie along the direction of approach to the target. From (6), we utilize $F^{i}(d)$ to compute the repulsive force $\mathbf{f}_{ang}(n)$ as a function of absolute angular difference $d_{ang}=|\psi_{t}^{T,B}-\psi_{t}^{T,S_{i}}|$. $\psi_{t}^{T,B}(n)$ is the angle of $B$ w.r.t to $T$, $\psi_{t}^{T,S_{i}}(n)$ is the angle of $S_{i}$ w.r.t $T$. We term this external control input as approach angle force (introduced in [19]). The total obstacle avoidance external control input on $B$ is given by,

[TABLE]

To embed this control input into the MPC dynamics we clamp the magnitude of $\mathbf{f}^{B}_{rep}(n)$ to $F_{max}$ (defined in (6)).

[TABLE]

Note that in practice, we observed that the external input $\mathbf{f}_{t}^{B}(n)$ could cause high-frequency oscillations in the horizon motion plans if $B$ is very close to obstacles. To reduce these oscillations, we add a fraction of external control input from the previous time step $\kappa\mathbf{f}_{t-1}^{B}(n),0<\kappa<1$. The total external input which is clamped at $F_{max}$ is now given by $\mathbf{f}_{t}^{B}(n)=\mathbf{f}_{t}^{B}(n)+\kappa\mathbf{f}_{t-1}^{B}(n)$. This operation recursively reduces the effect of obstacles on $B$ over multiple time steps and therefore ensures a smooth change in external control input.

Bounding-Box Deformations: The DVB can be deformed in size by regulating $r_{t}^{B}$ over the horizon. Reducing $r_{t}^{B}(n)$, instantaneously reduces the region of influence of the potential field. This enables $B$ to navigate through tight spaces. Fig. 4 highlights the $d_{min}$ of the potential field (dotted-circle) as the DVB navigates between obstacles. The deformation in $r_{t}^{B}$ is applied over the MPC horizon, and the rate of this deformation is proportional to the total repulsive potential field value $\mathbf{f}_{t}^{B}(n)$. In our work, the DVB is deformed only along its width. For ease of navigation of the mobile manipulators, it is important to restore $B$ to its original size in regions where obstacles are sparse. Therefore, $B$ is also associated with a small expansion potential field $f_{t}^{e}(n)$, which is directed towards increasing its width. This DVB width $w^{B}_{t}(n)$ and scale $r_{t}^{B}(n)$ over a horizon are computed as follows.

[TABLE]

where, $l^{B},w^{B}_{t}(n)$, refer to the length and variable width of the DVB respectively. $k_{s},k_{e}$ are positive constants with $k_{s}>k_{e}$. The region of influence of $f_{t}^{e}(n)$ is given by $d_{min}=min(r^{B}_{t})$ and $d_{max}=max(r^{B}_{t})$ which are the minimum and maximum allowed deformations respectively.

V-A4 MPC Formulation

[TABLE]

subject to,

[TABLE]

The objective of the MPC is given by (V-A1). The motion planning adheres to the following constraints.

LTI dynamics of $\mathbf{x}_{t}^{B}$ given by (14), 2. 2.

collision avoidance constraints incorporated as an external control input $\mathbf{f}_{t}^{B}\in\mathbb{R}^{2}$, 3. 3.

position and control saturation bounds on $\mathbf{x}_{t}^{B}$, $\mathbf{u}_{t}^{B}$.

Dynamics ($\mathbf{A}\in\mathbb{R}^{2\times 2}$) and control transfer ($\mathbf{B}\in\mathbb{R}^{2\times 2}$) matrices are given by $\mathbf{A}=\mathbf{I}_{2\times 2}$ and $\mathbf{B}=\Delta t\mathbf{I}_{2\times 2}$ where, $\mathbf{I}_{2\times 2}$ is an identity matrix and $\Delta t$ is the sampling time. The quadratic program computes optimal control inputs $\left[\mathbf{u}_{t}^{B*}(0)\cdots\mathbf{u}_{t}^{B*}(H)\right]$ and trajectory $\left[\mathbf{x}_{t}^{B*}(1)\cdots\mathbf{x}_{t}^{B*}(H+1)\right]$ towards the $\mathbf{x}_{t}^{B_{d}}$. The scale $r_{t}^{B}(1)$ is used as the initial DVB state for the next optimization iteration at $t+1$. The position $\mathbf{x}_{t}^{B}$ and yaw $\psi_{t}^{B}$ are then controlled using a proportional controller with $\mathbf{x}_{t}^{B*}(1)$ and $\psi_{t}^{T,B}$ (defined in (4)) as desired position and yaw.

V-B Payload Motion Planning

The DVB described in the previous section forms a multi-scale convex workspace for the system of robots and the payload over the MPC horizon. The primary goal of payload motion planning is to roll the payload $P$ to ensure that its projection $\hat{P}$ lies within the planned DVBs. This guarantees environmental collision avoidance for $P$. Accordingly, a non-linear optimization program with the objectives of minimizing payload roll $\phi_{t}$ and angular velocity $\omega_{t}$ over a planning horizon $H_{p}\leq H$ is formulated. The optimization is constrained to ensure $\hat{P}$ lies within the computed DVB for a horizon. It is important to note that the limit on $w_{t}^{B}$ is defined in accordance with the maximum roll $\phi_{max}$ of the payload. This guarantees the existence of a feasible solution to $\phi_{t}$. The constrained optimization is formulated as,

[TABLE]

subject to,

[TABLE]

where, $\text{w}_{1},\text{w}_{2}$ are constant positive weights on the objectives. Here, (17) defines linear polygonal regional constraints for the vertices of the payload $\forall n$. For each horizon step, $\mathbf{Reg}_{t}^{B}(n+1)\in\mathbb{R}^{4\times 2}$ and $\mathbf{b}_{t}^{B}(n+1)\in\mathbb{R}^{4}$ represent the planned DVBs. $T_{P}^{F}$ is a homogeneous transformation (in SE(3)) of a point $v_{i}$ (defined in the payload local frame) to the world frame ${F}$, for a roll of $\phi_{t}(n+1)$ and a translation of $\mathbf{x}_{t}^{P}$. $\hat{T}_{P}^{F}$ computes the projection of the transformed point onto the DVB. Here, $v_{i}$ are points which are defined in the payload local frame, to parameterize the payload. In our work, a cuboidal payload is used and parameterized using its vertices $v_{i},i\in[1,8]$. The extension to non-cuboidal payloads is straight-forward as long as its 3-D convex hull vertices are known. The constraint in (18) controls the time evolution of $\phi_{t}$, subject to limits on $\phi_{t}$ and $\omega_{t}$. The defined optimization minimizes the weighted sum of squares of $\phi_{t}$ and $\omega_{t}$ to ensure that payload rolls smoothly while staying within the planned DVBs (see Fig. 3). The optimal solution at the first horizon step (i.e. $\phi^{*}_{t}(1)$) is used as the roll for time instant $t$.

V-C Decentralized Formation Motion Planning

In this section we first introduce the inverse kinematics of the 6-dof manipulator used in this work and subsequently discuss the decentralized formation motion planning.

V-C1 6-dof Manipulator

The 6-revolute joints are visualized in Fig. 5(a). The joint configuration is similar to industrial 6-dof manipulators like the KR3 Agilus222http://www.kuka.com/ or the UR-3 manipulator333http://www.universal-robots.com/. The inverse kinematics (ikin) of the manipulator for a known grasp point $\mathbf{g}^{k}_{t}$ and mobile base position $\mathbf{x}^{k}_{t}$ is analytically determined as follows.

• •

$\theta_{t}^{1,k}$, is computed using the orientation of the vector ($\mathbf{g}^{k}_{t}-\mathbf{x}^{k}_{t}$).

• •

$\theta_{t}^{2,k},\theta_{t}^{3,k}$ form a planar two-link manipulator in the plane defined by $\theta_{t}^{1,k}$. $\theta_{t}^{3,k}=\cos^{-1}\frac{(R_{z}(\theta_{t}^{1,k})g^{k}_{t})_{x}^{2}+(R_{z}(\theta_{t}^{1,k})\mathbf{g}^{k}_{t})_{z}^{2}-l^{2}_{2}-l^{2}_{3}}{2l_{2}l_{3}}$. $l_{2}=\|m_{t}^{2,k}-m_{t}^{3,k}\|_{2}$, $l_{3}=\|m_{t}^{3,k}-m_{t}^{4,k}\|_{2}$ are link lengths and $R_{z}(\theta_{t}^{1,k})$ is a three dimensional rotation matrix about the first joint axis. $\theta_{t}^{2,k}=\tan^{-1}\frac{(R_{z}(\theta_{t}^{1,k})\mathbf{g}^{k}_{t})_{z}}{(R_{z}(\theta_{t}^{1,k})\mathbf{g}^{k}_{t})_{x}}-\tan^{-1}\frac{l_{3}\sin{\theta_{t}^{3,k}}}{l_{2}+l_{3}cos\theta_{t}^{3,k}}$.

• •

$\theta_{t}^{4-6,k}$ form a wrist configuration without any offset, whose values are determined based on the payload orientation. $\theta_{t}^{4,k},\theta_{t}^{5,k}$ are solved as spherical angles of the vector $(\mathbf{x}_{t}^{P}-\mathbf{g}_{t}^{k})$ and $\theta_{t}^{6,k}$ is along this vector, which is solved using the plane formed by the payload.

V-C2 Inter-Base Collision Avoidance

The planned DVB for a horizon using (13-16) form a collision-free convex hull for the robot formation. We define an operational regional constraint within $B$ for each mobile base as shown in Fig. 2(a). This region is undilated by the radius444A mobile base is defined as a cylinder of a certain radius and height of the robot to guarantee inter-mobile-base collision avoidance (visualized in Fig. 5(b)). For $K$ robots the bounding box for each horizon step $n$ is divided into $K$ equally sized smaller bounding boxes around $\mathbf{x}_{t}^{B}$ (virtual leader). Each smaller bounding box can be represented as a linear constraint to guarantee inter-base collision avoidance. To ensure manipulation feasibility and avoid singular configurations, the dimensions of the regional constraint bounding box are geometrically constrained by the link lengths of the manipulator, predefined height $h_{P}$ of the payload and the dimensions of the cuboidal payload.

V-C3 Inter-Manipulator Collision Avoidance

If a mobile base nears the edges of its regional-constraints, the manipulator elbow joints $m_{t}^{3,k}$ could collide with neighboring manipulators. To avoid this scenario, we communicate the position of $m_{t}^{3,k}$ with other robots $j\neq k$. We then associate a repulsive field $F^{i}(d_{elbow})$ (see (6)) as a function of inter-robot elbow distances $d_{elbow}$. Since the motion of base directly affects $\theta_{t}^{1,k}$ and $\mathbf{m}_{t}^{2-3,k}$, the repulsive field acting on each elbow joint directly translates to the mobile base. We incorporate this field as an external control input $\mathbf{f}_{t}^{k}$ for mobile base $k$ in MPC dynamics. The region of influence of this repulsive field is equal to $max(\|m_{t}^{2,k}-m_{t}^{3,k}\|_{2},~{}\|m_{t}^{3,k}-m_{t}^{4,k}\|_{2})$. This is highlighted using a dotted circle in Fig. 5(b).

V-C4 Cost

The goal of formation motion planning is not only to avoid collisions but also to ensure that the mobile bases minimize energy consumption and compute feasible trajectories over a prediction horizon $H_{r}\leq H$. We therefore minimize the robot control input for a horizon and guide the robots towards the center of their respective regional constraints over $H_{r}$. The cost is defined as,

[TABLE]

where, $\mathbf{u}^{k}_{t}(n)\in\mathbb{R}^{2}$ is the linear translational velocity control input for robot $k$ at control horizon step $n$, $\mathbf{x}^{k}_{ref}(n+1)$ is the trajectory of the center of the regional constraint of $k$ over the control horizon and $\boldmath{\Omega}_{\mathbf{u}},~{}\boldmath{\Omega}_{\mathbf{x}}$ are diagonal positive semi-definite weight matrices.

V-C5 Formation Trajectory Optimization

The optimal trajectory ($\mathbf{x}^{k*}_{t}\leftarrow\text{state},~{}\mathbf{u}^{k*}_{t}\leftarrow\text{control}$ ) for each robot $k$ is computed using a decentralized MPC given by,

[TABLE]

subject to,

[TABLE]

Here, (22) is the linear translational dynamics for robot $k$. $\mathbf{f}_{t}^{k}$ embeds the inter-manipulator obstacle avoidance constraint and is constant over the horizon $H_{r}$. Input $\mathbf{f}_{t}^{k}$ is clamped to $0.5\|\mathbf{u}^{k}_{max}\|$ and $\mathbf{u}^{k}_{max}\geq\mathbf{u}^{B}_{max}$. The DMPC is constrained to choose the $\mathbf{u}_{t}^{k}$ in the range of $0.5\mathbf{u}^{k}_{\mathrm{min}}\leq~{}\mathbf{u}_{t}^{k}(n)\leq 0.5\mathbf{u}^{k}_{\mathrm{max}}$ for $\|\mathbf{f}_{t}^{k}\|>0$. This ensures that $\mathbf{f}_{t}^{k}$ does not force the robot out of its defined regional constraint. However, if $\|\mathbf{f}_{t}^{k}\|=0$ at $t$, then the MPC controls the full range of $\mathbf{u}_{t}^{k}$. Eq. (23) is a linear regional constraint for robot $k$ over the control horizon where, $\mathbf{Reg}^{k}_{t}(n+1)\in\mathbb{R}^{4\times 2}$ and $\mathbf{b}^{k}_{t}(n+1)\in\mathbb{R}^{4}$. $\mathbf{u}_{t}^{k*}(0)$ is used as control input for mobile base $k$ at time $t$. A feasible inverse kinematics solution can be computed using $\mathbf{g}_{t}^{k}$ and $\mathbf{x}^{k*}_{t}(1)$ and the equations mentioned in Sec. V-C1.

VI Results and Discussion

In this section, we detail the results of our proposed motion planning algorithm. The DVB MPC and the decentralized formation MPC are both numerically solved using the operator splitting quadratic program (OSQP) solver [20]. The payload motion planning is solved as a sequential quadratic program. The proposed algorithm is implemented in Matlab and runs on an Intel i7-7700HQ CPU. The algorithm is validated using multiple simulation environments varying in, (i) the number of static obstacles and dynamic obstacles, (ii) the number of robots, (iii) the dimensions of payloads, and, (iv) trajectories of a moving target. These results are best viewed in the video https://youtu.be/9EKj7RwRs_4. For the sake of analysis and brevity, we investigate and discuss one of these environments in detail.

We consider an environment having three closely spaced, static obstacles and four randomly moving dynamic obstacles. A target $T$ navigates at approximately $0.4~{}ms^{-1}$ to provide high-level motion goals to the system. Six mobile manipulators are tasked with transporting a payload of dimensions $3~{}m\times 3~{}m\times 0.1~{}m$ through this environment while tracking $T$. Fig. 6 showcases top and isometric views as the system navigates through tight spaces, using the proposed hierarchical motion planning algorithm. The dashed black line is the trajectory followed by $T$. The three static obstacles are visualized as pink cylinders, and the four dynamic obstacles are visualized as black cylinders. Green lines highlight the trajectories of dynamic obstacles. The motion of the system is best visualized in the following video https://youtu.be/9EKj7RwRs_4. Dynamic obstacles are adversarial and hinder the motion of the system as observed in Fig. 6 and the video.

*Deformable Virtual Bounding Box: * The state and control limits are $[-30,-30]\leq~{}\mathbf{x}_{t}^{B}~{}\leq[30,30]$ in $m$ and $[-2,-2]~{}\leq\mathbf{u}_{t}^{B}~{}\leq[2,2]$ in $ms^{-1}$ respectively. The control horizon $H$ has an effect on real time performance. For $H=12$ time steps and $\Delta t=0.1~{}s$, the average execution time $t_{avg}=0.17~{}s$. $d_{max}=d_{min}+1.8~{}m$ defines $\mathbf{f}_{t}^{B}$’s region of influence with an $F_{max}=2.5~{}ms^{-1}$. The limits on DVB scale $1.3~{}m\leq r_{t}^{B}\leq 2.12~{}m$ are defined using the projection of payload for $\phi_{t}=0$ and $\phi_{t}=\phi_{max}$ respectively. In Fig. 6 the red rectangles shows the motion of the DVB for $H$ time steps. The payload (blue rectangle) orients itself to lie within the defined workspace. Each mobile base (blue cylinders) operates within its regional constraints (small pink rectangles) and each manipulator (in black) has a collision-free kinematically feasible configuration. Fig. 7(a-d) showcases the DVB pose $\mathbf{x}_{t}^{B}=[x_{t}^{B}~{}y_{t}^{B}~{}\psi_{t}^{B}]$ and scale $r_{t}^{B}$ over the experiment duration. Notice that Fig. 6(a) and Fig. 7 are divided into three smaller timelines $\tau_{1},~{}\tau_{2},~{}\tau_{3}$. In $\tau_{1}$, the DVB deforms from $r_{t}^{B}=2.12~{}m$ to $r_{t}^{B}=2~{}m$ to avoid a dynamic obstacle and subsequently expands for about $10~{}s$. In $\tau_{2}$, the DVB navigates through a narrow gap between static obstacles $S_{1},~{}S_{2}$ causing a decrease in $r_{t}^{B}$. Next, it encounters multiple dynamic obstacles causing a sharp decrease to $r_{t}^{B}=1.75~{}m$, as observed in Fig. 7(d). In $\tau_{3}$, $r_{t}^{B}$ increases to $1.92~{}m$ until the DVB performs a sharp left turn around $S_{3}$ to keep track of $T$. Finally, $r_{t}^{B}$ gradually increases $2.12~{}m$ as obstacles become sparse. The repulsive fields $\mathbf{f}_{sta}$, $\mathbf{f}_{dyn}$ aid in maintaining a distance of at least $r_{t}^{B}$ w.r.t static and dynamic obstacles throughout $\tau_{1},~{}\tau_{2}$ and $~{}\tau_{3}$. In obstacle-free regions, $f_{t}^{e}$ restores $r_{t}^{B}$ to $r_{max}$. In Fig.7(a,b) the three static obstacle positions $S_{1},S_{2},S_{3}$ are overlaid (in pink). Observe that $x_{t}^{B}$ and $y_{t}^{B}$ simultaneously do not intersect the same pink line $(S_{i})$ at the same exact time instant. This validates static obstacle avoidance.

*Payload Motion Planning: * The limits on motion planning are $0~{}\leq~{}\phi_{t}~{}\leq\frac{2\pi}{5}$ in $rad$ and $-1\leq~{}\omega_{t}(n)~{}\leq 1$ in $rads^{-1}$. For a control horizon $H_{p}=5$ and $\Delta t=0.1~{}s$, $t_{avg}=0.05~{}s$ was observed. The payload’s roll $\phi_{t}$ varies in accordance with $r_{t}^{B}$ using non-linear optimization of Sec. V-B. Fig. 7(e,f) plots the variation of $\phi_{t}$ and $\omega_{t}$. We observe that the payload rolls in accordance with $r_{t}^{B}$ and varies smoothly along the trajectory thereby validating the proposed optimization.

*Decentralized Formation Motion Planning: * Each mobile base is a cylinder of radius $0.2~{}m$ and height $0.2~{}m$. The control limits of the mobile base are $-4\leq~{}\mathbf{u}_{t}^{k}~{}\leq 4$ in $ms^{-}1$. A $t_{avg}=0.08~{}s$ was observed for $H_{r}=5$. In this experiment, $m_{t}^{1,k}=m_{t}^{2,k}$ and $m_{t}^{4-6,k}=g_{t}^{k}$ (co-axial wrist). The link lengths are, $\|m_{t}^{2,k}-m_{t}^{3,k}\|_{2}=1.316~{}m$, $\|m_{t}^{3,k}-m_{t}^{4,k}\|_{2}=1.484~{}m$. The results of decentralized robot motion planning are showcased in Fig. 8(a-d). The trajectories $\mathbf{x}_{t}^{k}=[x_{t}^{k}~{}y_{t}^{k}]$ are observed to be centered around $\mathbf{x}_{t}^{B}$ (virtual formation leader) and vary smoothly over time. The control inputs $\mathbf{u}_{t}^{k}$ vary gradually over the trajectory and lie within the limits defined by the MPC. The high frequency changes in $\mathbf{u}_{t}^{k}$ around $100~{}s$ ($\tau_{2}$), in Fig. 8(c,d), can be attributed to low inter-manipulator spacing due to low $r_{t}^{B}$ (see Fig. 7(d)), leading to high $\mathbf{f}_{t}^{k}$. Fig. 8(e-j) showcase the variation in the joint angles of manipulators over time. Notice that the variation is smooth and the velocities of the joints lie in the range of $[-1~{}rads^{-1},1~{}rads^{-1}]$. Note that the variations in $\theta_{t}^{4}$ are just angle wraparounds. These plots validate the efficacy of the proposed decentralized formation controller.

VII Conclusions and Future Work

In this paper, a novel kinematic motion planning algorithm for cooperative mobile non-planar payload manipulation in dynamic environments is presented. Three constrained optimization problems are formulated to handle key challenges namely, (1) computationally scalable goal-directed non-planar manipulation, (2) environmental obstacle avoidance, and, (3) inter-robot obstacle avoidance. In future, we plan to physically validate the algorithm on real robots.

Acknowledgments

We thank Eric Price for his valuable advice during the course of this work.