Steering a Particle Swarm Using Global Inputs and Swarm Statistics

TL;DR

This paper explores controlling large particle swarms using only global inputs and swarm statistics, demonstrating controllability, developing controllers, and validating on kilobots for manipulation tasks.

Contribution

It introduces controllers based on mean and variance for swarm manipulation, proves controllability, and implements these controllers on kilobots for practical tasks.

Findings

Mean position is controllable in swarm systems.

Variance can be controlled under certain conditions.

Controllers successfully manipulated kilobots using global inputs.

Abstract

Microrobotics has the potential to revolutionize many applications including targeted material delivery, assembly, and surgery. The same properties that promise breakthrough solutions---small size and large populations---present unique challenges for controlling motion. Robotic manipulation usually assumes intelligent agents, not particle systems manipulated by a global signal. To identify the key parameters for particle manipulation, we used a collection of online games where players steer swarms of up to 500 particles to complete manipulation challenges. We recorded statistics from over ten thousand players. Inspired by techniques where human operators performed well, we investigate controllers that use only the mean and variance of the swarm. We prove the mean position is controllable and provide conditions under which variance is controllable. We next derive automatic controllers for these and a hysteresis-based switching control to regulate the first two moments of the particle distribution. Finally, we employ these controllers as primitives for an object manipulation task and implement all controllers on 100 kilobots controlled by the direction of a global light source.

Equations

$${\dot{x}}_i$$

$${\dot{y}}_i$$

$$\dot{\mathbf{x}}(t)$$

$$\left[\begin{array}{c}\dot{x}\\ \ddot{x}\\ \dot{y}\\ \ddot{y}\end{array}\right]=\left[\begin{array}{cccc}0& 1& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 1\\ 0& 0& 0& 0\end{array}\right]\left[\begin{array}{c}x\\ \dot{x}\\ y\\ \dot{y}\end{array}\right]+\left[\begin{array}{cc}0& 0\\ \frac{1}{m}& 0\\ 0& 0\\ 0& \frac{1}{m}\end{array}\right]\left[\begin{array}{c}{u}_x\\ u_y\end{array}\right].$$

$$\mathcal{C}$$

$$\text{Here }\mathcal{C}$$

$$\text{rank}(\mathcal{C})$$

$$\left[\begin{array}{c}{\dot{x}}_1\\ {\ddot{x}}_1\\ ⋮\\ {\dot{x}}_n\\ {\ddot{x}}_n\text{par}\end{array}\right]=\left[\begin{array}{ccccc}0& 1& \dots & 0& 0\\ 0& 0& \dots & 0& 0\\ ⋮& ⋮& \ddots & ⋮& ⋮\\ 0& 0& \dots & 0& 1\\ 0& 0& \dots & 0& 0\end{array}\right]\left[\begin{array}{c}{x}_1\\ {\dot{x}}_1\\ ⋮\\ x_n\\ {\dot{x}}_n\end{array}\right]+\left[\begin{array}{c}0\\ 1\\ ⋮\\ 0\\ 1\end{array}\right]u_x.$$

$${\mathcal{C}}_n=\left[\begin{array}{c}0\\ 1\\ ⋮\\ 0\\ 1\end{array}\left|\begin{array}{c}1\\ 0\\ ⋮\\ 1\\ 0\end{array}\right|\begin{array}{c}0\\ 0\\ ⋮\\ 0\\ 0\end{array}\left|\begin{array}{c}0\\ 0\\ ⋮\\ 0\\ 0\end{array}\right|\dots \right],\text{rank}({\mathcal{C}}_n)=2.$$

$$\left[\begin{array}{c}\dot{\bar{x}}\\ \ddot{\bar{x}}\end{array}\right]$$

$$+\frac{1}{n}\left[\begin{array}{ccccc}0& 0& \dots & 0& 0\\ 0& 1& \dots & 0& 1\end{array}\right]\left[\begin{array}{c}0\\ 1\\ ⋮\\ 0\\ 1\end{array}\right]u_x.$$

$$\left[\begin{array}{c}\dot{\bar{x}}\\ \ddot{\bar{x}}\end{array}\right]=\left[\begin{array}{cc}0& 1\\ 0& 0\end{array}\right]\left[\begin{array}{c}\bar{x}\\ \dot{\bar{x}}\end{array}\right]+\left[\begin{array}{c}0\\ 1\end{array}\right]u_x.$$

$${\mathcal{C}}_{\mu }=\left[\begin{array}{c}0\\ 1\end{array}|\begin{array}{c}1\\ 0\end{array}\right],\text{rank}({\mathcal{C}}_{\mu })=2.$$

$$u_x$$

$$u_y$$

$$\overline{x}(\mathbf{x})=\frac{1}{n}\sum _{i=1}^n{x}_i,{\sigma }_x^2(\mathbf{x})$$

$$\overline{y}(\mathbf{x})=\frac{1}{n}\sum _{i=1}^n{y}_i,{\sigma }_y^2(\mathbf{x})$$

$$\dot{\mathbf{x}}(t)$$

$${\dot{\sigma }}_x^2(\mathbf{x}(t),\mathbf{u}(t))=W\mathbf{\epsilon },{\sigma }_x^2(t)={\sigma }_x^2(0)+W\mathbf{\epsilon }t.$$

$$[{\sigma }_x^2,{\sigma }_y^2]=\frac{1}{12}[({\ell }_x-2r{)}^2,({\ell }_y-2r{)}^2].$$

$${\sigma }_{\mathrm{optimal}}^2(n,r)\approx \frac{\sqrt{3}}{\pi }n{r}^2\approx 0.55n{r}^2.$$

$$V(t,\mathbf{x})$$

$$\dot{V}(t,\mathbf{x})$$

$$u(t)$$

$${\dot{\sigma }}^2(\mathbf{x})$$

$$u_x$$

$$u_y$$

$$x_{\mathrm{goal}}({\sigma }_{\mathrm{ref}}^2)={\ell }_x/2=r+\sqrt{3{\sigma }_{\mathrm{ref}}^2}.$$

$${\sigma }_{\min }^2$$

$${\sigma }_{\max }^2$$

$$r(x,\mathbf{u})$$

$$\mathbf{D}(x)=\arg \underset{\mathbf{u}}{\max }[r(x,\mathbf{u})+\sum _{j=1}^N\hat{V}(x_j)p(x_j|x,\mathbf{u})].$$

$$i=1$$

$$\hat{V}(x_j)=\gamma \underset{\mathbf{u}}{\max }[r(x_j,\mathbf{u})+\sum _{i=1}^N\hat{V}(x_i)p(x_i|x_j,\mathbf{u})]$$

$$F_{\mathrm{att}}$$

Full text

Robotic manipulation usually assumes intelligent agents, not particle systems manipulated by a global signal. To identify the key parameters for particle manipulation, we used a collection of online games where players steer swarms of up to 500 particles to complete manipulation challenges. We recorded statistics from over ten thousand players. Inspired by techniques where human operators performed well, we investigate controllers that use only the mean and variance of the swarm. We prove the mean position is controllable and provide conditions under which variance is controllable. We next derive automatic controllers for these and a hysteresis-based switching control to regulate the first two moments of the particle distribution. Finally, we employ these controllers as primitives for an object manipulation task and implement all controllers on 100 kilobots controlled by the direction of a global light source.

I Introduction

Large populations of micro- and nanorobots are being produced in laboratories around the world, with diverse potential applications in drug delivery and construction, see [1, 2, 3]. These activities require robots that behave intelligently. Limited computation and communication rules out autonomous operation or direct control over individual units; instead we must rely on global control signals broadcast to the entire particle population. This paper examines object manipulation by a swarm of particles, each actuated by the same shared global input, as illustrated in Fig. 1. The transportation methodology is similar to that in [4], but rather than using onboard computation or sensing, the particles all move in the same direction.

Many promising applications for particle swarms require direct human control, but user interfaces to thousands—or millions—of particles is a daunting human-swarm interaction (HSI) challenge. Our early work with over a hundred hardware robots and thousands of simulated particles demonstrated that direct human control of large swarms is possible, [5]. Unfortunately, the logistical challenges of repeated experiments with over one hundred robots prevented large-scale tests. There is currently no comprehensive understanding of user interfaces for controlling multi-robot systems with massive populations. One contribution of this paper is a tool for investigating HSI methods through statistically significant numbers of experiments.

Often particles are difficult or impossible to sense individually due to their size and location. For example, microrobots are smaller than the minimum resolution of a clinical MRI-scanner, see [6], however it is often possible to sense global properties of the group such as mean position and variance. To make progress in automatic control with global inputs, this paper presents swarm manipulation controllers inspired by our online experiments that require only mean and variance measurements of the particle’s positions. To perform the object manipulation task illustrated in Fig. 1, we use these controllers as primitives, policy iteration for path planning, handle outliers by partitioning the workspace, and minimize pushing the object backwards with potential field navigation.

Our paper is organized as follows. After a discussion of related work in §II, we describe our experimental methods for an online human-user experiment and their results in §III. Next we prove that the mean and variance of a particle swarm are controllable in §IV, and present automatic controllers in §V. We use these controllers as primitives and present a framework for manipulating an object through a maze in §VI. We conclude with implementations of these controllers in our hardware robots and use them to complete an object manipulation task with 100 kilobots in §VII.

II Related Work

This section describes global control challenges and reviews highlights of human-swarm interaction, block pushing, and compliant manipulation.

II-A Global control of microrobots

This paper investigates global control of particles that have no onboard computation. This prevents us from applying controllers that require computation on the agents, as in [7, 8, 9]. Another control paradigm is to construct robots with physical heterogeneity so that they respond differently to a global broadcast control signal. Examples include scratch-drive microrobots, actuated and controlled by a DC voltage signal from a substrate by [10, 11]; magnetic structures with different cross-sections that can be independently steered by [12, 13]; MagMite microrobots with different resonant frequencies and a global magnetic field by [14]; and magnetically controlled nanoscale helical screws constructed to stop movement at different cutoff frequencies of a global magnetic field by [15] and [1]. Similarly, our previous work, [16, 17], focused on exploiting inhomogeneity between robots. These control algorithms theoretically apply to any number of robots, even robotic continuums.

However, process noise cancels the differentiating effects of inhomogeneity for more than tens of robots. We desire control algorithms that extend to many thousands of robots. Limited position control was achieved by [18] and our previous work [19], but both used robots commanded in their local coordinate frame. Our new submission focuses on a more common paradigm: particles commanded in a global coordinate frame

While it is now possible to create many microrobots, there remain challenges in control, sensing, and computation:

Control—global inputs

Many micro- and nanorobotic systems, see [15, 2, 3, 10, 11, 20, 12, 13, 14, 1] rely on global inputs, where each robot receives an exact copy of the control signal. Our experiments follow this global model.

Sensing—large populations

$n$ differential-drive robots in a 2D workspace require $3n$ state variables. Even holonomic robots require $2n$ state variables. Numerous methods exist for measuring this state in microrobotics [1, 3, 6]. These solutions use computer vision systems to sense position and heading angle, with corresponding challenges of handling missed detections and image registration between detections and robots. These challenges increase at small scales where sensing competes with control for communication bandwidth. We examine control when the operator has access to partial feedback, including only the first and/or second moments of a population’s position, or only the convex-hull containing the robots.

Computation—calculating the control law

In our previous work the controllers required at best a summation over all the robot states, see [17] and at worst a matrix inversion, see [16]. These operations become intractable for large populations of robots. By focusing on human control of large robot populations, we accentuate computational difficulties because the controllers are implemented by the unaided human operator.

II-B Human-swarm interaction

Most humans are able to, with practice, steer a swarm of robots controlled by a global input. Prior to our paper, no algorithm existed. Using human input to learn how to control a dynamic system is a line of research with a rich history [21, 22]. This paper exploits insights gained from SwarmControl.net, particularly the fact that having a swarm’s mean and variance is sufficient for object manipulation through an obstacle field.

A user interface enabling an operator to maneuver a swarm of robots through a cluttered workspace by specifying the bounding prism for the swarm and then translating or scaling this prism is designed in [23]. Our paper shares the concept of a global control input, but our robots have no onboard computation and cannot track a virtual boundary.

Human fanout, the number of robots a single human user could directly control is studied in [24]. They postulated that the optimal number of robots was approximately the autonomous time divided by the interaction time required by each robot. Their sample problem involved a multi-robot search task, where users could assign goals to robots. Their user interaction studies with simulated planar robots indicated a fanout plateau of about 8 robots, with diminishing returns for more robots. They hypothesized the location of this plateau is highly dependent on the underlying task. Indeed, our paper indicates there are tasks without plateaus. Their research investigated robots with 3 levels of autonomy. We use robots without autonomy, corresponding with their first-level robots.

Several user studies compare methods for controlling large swarms of simulated robots, for example [25, 26, 27]. These studies provide insights but are limited by cost to small user studies; have a closed-source code base; and focus on controlling intelligent, programmable agents. For instance, the studies [25], [26], and [27] were limited to a pool of 5, 18, and 32 participants. Using an online testing environment, we conduct similar studies but with sample sizes three orders of magnitude larger.

II-C Block pushing and compliant manipulation

Unlike caging manipulation, where robots form a rigid arrangement around an object, as in [28, 29], our swarm of robots is unable to grasp the blocks they push, and so our manipulation strategies are similar to nonprehensile manipulation techniques, e.g. [30], where forces must be applied along the center of mass of the moveable object. A key difference is that our robots are compliant and tend to flow around the object, making this similar to fluidic trapping as in [31] and [32].

Our $n$-robot system with 2 control inputs and 4$n$ states is inherently under-actuated, and superficially bears resemblance to compliant, under-actuated manipulators. Our swarms conform to the object to be manipulated, but lack the restoring force provided by flexures in [33] or silicone in [34]. Our swarms tend to disperse and so to regroup them we require artificial forces like the variance control primitives in §IV-C.

II-D Relationship to authors’ prior work

This paper combines the content of two preliminary conference papers, extending their substance and providing full details in a single journal paper. One paper covered the first three months of SwarmControl.net experiments [35], and the second presented simulations of object manipulation [36]. This paper presents three years of results from SwarmControl.net. For object manipulation, this paper presents robust new algorithms for manipulation, path planning, and obstacle avoidance, and a rich set of parameter sweeps over key variables. All hardware validation experiments are new.

III Online experiment

The goal of these online experiments is to test several scenarios involving large-scale human-swarm interaction (HSI), and to do so with a statistically-significant sample size. Towards this end, we have created SwarmControl.net: an open-source, online testing platform suitable for inexpensive deployment and data collection on a scale not yet seen in swarm robotics research. Screenshots from this platform are shown in Fig. 2. All code and experimental results are online at [37].

We developed a flexible testing framework for online human-swarm interaction studies. Over 5,000 humans performed over 20,000 swarm-robotics experiments with this framework, logging almost 700 hours of experiments. These experiments indicated three lessons used for designing automatic controllers for object manipulation with particle swarms:

1. When the number of particles is large ($>50$) varying the number of particles does not significantly affect the performance.

2. Swarm control is robust to IID noise.

3. Controllers that only use the mean and variance of the swarm can perform better than controllers with full feedback.

III-A Implementation

Our web server generates a unique identifier for each participant and sends it along with the landing page to the participant. A script on the participant’s browser runs the experiment and posts the experiment data to the server.

Anonymized human subject data was collected under IRB #14357-01.

We designed six experiments to investigate human control of large swarms for manipulation tasks. Screenshots of representative experiment are shown in Fig. 2. Each experiment examined the effects of varying a single parameter: population of particles for manipulation, four levels of visual feedback, different levels of Brownian noise.The users could choose which experiment to try, and our architecture randomly assigned a parameter value for each trial. We recorded the completion time and the participant ID for each successful trial.

III-B Varying number

This experiment varied from 1 to 500 the population of particles used to transport an object. The total area, maximum particle speed, and total net force the swarm could produce were constant. The particles pushed a large hexagonal object through an S-shaped maze. We hypothesized participants would complete the task faster with more particles. The results, shown in Fig. 3, do not support our hypothesis, indicating a minimum around 130 particles, but only a gradual increase in completion time from 50 to 500.

III-C Varying visualization

This experiment explores manipulation with varying amounts of sensing information: full-state sensing provides the most information by showing the position of all particles; convex-hull draws a convex hull around the outermost particles; mean provides the average position of the population; and mean + variance adds a confidence ellipse. Fig. 4 shows screenshots of the same particle swarm with each type of visual feedback. Full-state requires $2n$ data points for $n$ particles. Convex-hull requires at worst $2n$, but according to [38], the expected number is $O(2n^{1/3})$. Mean requires two, and variance three, data points. Mean and mean + variance are convenient even with millions of particles.

Our hypothesis predicted a steady decay in performance as the amount of visual feedback decreased. Our experiment indicated the opposite: players with just the mean completed the task faster than those with full-state feedback. As Fig. 5.b shows, the levels of feedback arranged by increasing completion time are [mean, mean + variance, full-state, convex-hull]. All experiments lasting over 300s were removed, under the assumption that the user stopped playing. Using ANOVA analysis, we rejected the null hypothesis that all visualization methods are equivalent, with $p$-value $2.69\times 10^{-19}$. Anecdotal evidence from beta-testers who played the game suggests that tracking 100 particles is overwhelming—similar to schooling phenomenons that confuse predators—while working with just the mean + variance is like using a “spongy” manipulator. Our beta-testers described convex-hull feedback as confusing and irritating. A single particle left behind an obstacle will stretch the entire hull, obscuring the majority of the swarm. Similarly, our algorithm must be robust to outliers.

III-D Varying noise

This experiment varied the strength of disturbances to study how noise affects human control of large swarms. Noise was applied at every timestep:

[TABLE]

$m_{i},\psi_{i}$ were uniformly IID, with $m_{i}\in[0,M]$ and $\psi_{i}\in[0,2\pi]$. $M$ was a constant for each trial ranging from 0 to 200% of the maximum control power ($u_{\rm{max}}$).

We hypothesized 200% noise was the largest a human could be expected to control—at 200% noise, the particles move erratically. Disproving our hypothesis, the results in Fig. 6.a show only a 40% increase in completion time for the maximum noise. This indicates swarm control is robust to IID noise.

IV Global Control Laws for a holonomic swarm

Emboldened by the three lessons from our online experiments, this section presents automatic controllers for large numbers of particles that only rely on the first two moments of the swarm position distribution.

We represent particles as holonomic robots that move in the 2D plane. We want to control position and velocity of the particles. First, assume a noiseless system containing one robot with mass $m$. Our inputs are global forces $[u_{x},u_{y}]$. We define our state vector $\mathbf{x}(t)$ as the $x$ position, $x$ velocity, $y$ position and $y$ velocity. The state-space representation in standard form is:

[TABLE]

and our state space representation is:

[TABLE]

We want to find the number of states that we can control, which is given by the rank of the controllability matrix

[TABLE]

and thus all four states are controllable. This section starts by proving independent position control of many robots is not possible, but the mean position can be controlled. We then provide conditions under which the variance of many robots is also controllable.

IV-A Independent control of many particles is impossible

In this model, a single particle is fully controllable. For holonomic robots, movement in the $x$ and $y$ coordinates are independent, so for notational convenience without loss of generality we will focus only on movement in the $x$ axis. Given $n$ particles to be controlled in the $x$ axis, there are $2n$ states: $n$ positions and $n$ velocities. Without loss of generality, assume $m=1$. Our state-space representation is:

[TABLE]

However, just as with one particle, we can only control two states because the controllability matrix $\mathcal{C}_{n}$ has rank two:

[TABLE]

IV-B Controlling the mean position

This means any number of particles controlled by a global command have just two controllable states in each axis. We cannot arbitrarily control the position and velocity of two or more robots, but have options on which states to control. We create the following reduced order system that represents the mean $x$ position and velocity of the $n$ particles:

[TABLE]

Thus:

[TABLE]

We again analyze the controllability matrix $\mathcal{C}_{\mu}$:

[TABLE]

Thus the mean position and mean velocity are controllable.

There are several techniques for breaking the symmetry of the control control input to allow controlling more states, for example by using obstacles as in [5], or by allowing independent noise sources as in [19].

We control mean position with a PD controller that uses the mean position and mean velocity. $[u_{x},u_{y}]^{\top}$ is the global force applied to each robot:

[TABLE]

$K_{p}$ is the proportional gain, and $K_{d}$ is the derivative gain.

IV-C Controlling the variance

The variance, $\sigma_{x}^{2},\sigma_{y}^{2}$, of $n$ robots’ position is computed as:

[TABLE]

Controlling the variance requires being able to increase and decrease the variance. We will list a sufficient condition for each. Microscale systems are affected by unmodelled dynamics. These unmodelled dynamics are dominated by Brownian noise, described by [39]. To model this (1) must be modified as follows:

[TABLE]

where $W\bm{\varepsilon}(t)$ is a random perturbation produced by Brownian noise with magnitude $W$. Given a large obstacle-free workspace with $\mathbf{u}(t)=0$, a Brownian noise process increases the variance linearly with time.

[TABLE]

If faster dispersion is needed, the swarm can be pushed through obstacles such as a diffraction grating or Pachinko board as in [5].

If robots with radius $r$ are in a bounded environment with sides of length $[\ell_{x},\ell_{y}]$, the unforced variance asymptotically grows to the variance of a uniform distribution,

[TABLE]

A flat obstacle can be used to decrease variance. Pushing a group of dispersed robots against a flat obstacle will decrease their variance until the minimum-variance (maximum density) packing is reached. For large $n$,  [40] showed that the minimum-variance packing for $n$ circles with radius $r$ is

[TABLE]

We will prove the goal is globally asymptotically stabilizable by using a control-Lyapunov function, as in [41]. A suitable Lyapunov function is the squared variance error:

[TABLE]

We note here that $V(t,\mathbf{x})$ is positive definite and radially unbounded, and $V(t,\mathbf{x})\equiv 0$ only at $\sigma^{2}(\mathbf{x})=\sigma^{2}_{\rm{goal}}$. To make $\dot{V}(t,\mathbf{x})$ negative semi-definite, we choose

[TABLE]

For such a $u(t)$,

[TABLE]

and thus $\dot{V}(t,{\bf x})$ is negative definite and the variance is globally asymptotically stabilizable.

A PID controller to regulate the variance to $\sigma^{2}_{\rm{ref}}$ is:

[TABLE]

We call the gain scaling the variance error $K_{i}$ because the variance, if unregulated, integrates over time. Eq. (20) assumes the nearest wall is to the left of the robot at $x=0$, and chooses a reference goal position that in steady-state would have the correct variance according to (15):

[TABLE]

If a wall to the right is closer, the signs of $[K_{p},K_{i}]$ are inverted, and the location $x_{\rm{goal}}$ is translated.

IV-D Controlling both mean and variance

The mean and variance of the swarm cannot be controlled simultaneously, however if the dispersion due to Brownian motion is less than the maximum controlled speed, we can adopt the hybrid, hysteresis-based controller shown in Alg. 1 to regulate the mean and variance. Such a controller normally controls the mean position, but switches to minimizing variance if the variance exceeds some $\sigma_{\rm{max}}^{2}$. Variance is reduced until less than $\sigma_{\rm{min}}^{2}$, then control again regulates the mean position. This technique satisfies control objectives that evolve at different rates as in [42], and the hysteresis avoids rapid switching between control modes. The process is illustrated in Fig. 7.

A key challenge is to select proper values for $\sigma_{\rm{min}}^{2}$ and $\sigma_{\rm{max}}^{2}$. The optimal packing variance was given in (16). The random packings generated by pushing our robots into corners are suboptimal, so we choose the conservative values:

[TABLE]

V Simulation of control laws

Our simulations use a Javascript port of Box2D, a popular 2D physics engine with support for rigid-body dynamics and fixed-time step simulation, presented in [43]. All experiments in this section ran on a Chrome web browser on a 2.6 GHz Macbook. All code is available at [44].

V-A Controlling the mean position

We performed a parameter sweep using the PD controller (11) to identify the best control gains. Representative experiments are shown in Fig. 8. 100 particles were used and the maximum speed was 3 meters per second. As shown in Fig. 8, we can achieve an overshoot of 1% and a rise time of 1.52 s with $K_{p}=4$, and $K_{d}=1$.

V-B Controlling the variance

For variance control we use the control law (20). Results are shown in Fig. 9, with $K_{p,i,d}=[4,1,1]$.

V-C Hybrid control of mean and variance

Fig. 10 shows a simulation run of the hybrid controller in Alg. 1 with 100 particles in a square workspace containing no internal obstacles.

VI Particle Swarm Object manipulation

This section analyzes an object manipulation task attempted by our hybrid, hysteresis-based controllers. The swarm must deliver the object to the goal region. To solve this object manipulation task we divide the task into three components: 1) designing a policy for the object, 2) pushing the object with a compliant swarm, and 3) managing outliers.

The table below summarizes the simulation results for each 10 successful trials:

[TABLE]

VI-A Learning a policy for the object

To design the policy we first discretize the environment. In [36], we used breadth-first search (BFS) on this discretized grid, but using workspace BFS fails to account for the hull of the object and will suggest moves that can cause collisions with the workspace. A configuration-space BFS approach avoids that problem but still fails to model uncertain actuation of the object by the swarm.

To solve both these problems, this paper models object movement as a Markov Decision Process (MDP) with non-deterministic movement. Value iteration, as described in [45], is used to learn an optimal policy. At each state the object can be commanded to move in one of eight directions with a small probability of moving in a wrong direction.

The reward function $r(x,\mathbf{u})$ is defined as

[TABLE]

where $x$ is the current state and $\mathbf{u}$ is the action. Value iteration computes $\hat{V}(x)$, the expected discounted sum reward if the optimal policy is implemented, for the object starting in each state $x$. The optimal policy is

[TABLE]

The value function $\hat{V}(x_{j})$ is calculated by computing the value $\hat{V}$ for all $N$ states and iterating until convergence:

[TABLE]

In our experiments $\gamma=0.97$, and (VI-A) was iterated 200 times. A Matlab implementation of this algorithm is available at [46].

$\mathbf{M}_{\rm{BFS}}$ and the value function are shown in Fig. 11. In 10 simulations with 100 particles, pushing the object to goal using BFS required 183$\pm$179 s while Policy Iteration required 90$\pm$35 s (mean$\pm$std).

VI-B Potential fields for swarm management with a compliant manipulator

When the swarm is in front of the object, control law (11) pushes the object backwards. To fix this, we implement a potential field approach inspired by [47] that attracts the swarm to the intermediate goal, but repulses the swarm from in front of the object. The repulsive potential field is centered at the object’s COM and is active for a radius $\rho_{0}$, but is implemented only when the swarm mean is within $\theta$ of the desired direction of motion $\mathbf{D}(\mathbf{b})$ as shown in Fig. 12b.

[TABLE]

In simulations, $\theta=\pi/2$, $\eta=75$, $\zeta=2$ and $\rho_{0}=3$. Because the kilobots have a slower time constant, they use $\theta=\pi/2$, $\eta=50$, $\zeta=1$ and $\rho_{0}=7.5$.

In 10 simulations with 100 particles, pushing the object to goal without a repulsive potential field failed in two of twelve runs. No failures occurred with the repulsive potential field. Of successful trials, completion time without repulsive potential fields required 245$\pm$135 s while using repulsive potential fields required 90$\pm$35 s (mean$\pm$std).

VI-C Outlier rejection

The variance controller in Alg. 1 is a greedy algorithm that is susceptible to outliers. The controller in [36] failed in $14\%$ trials, some particles were unable to reach the object because workspace obstacles were blocking them. This failure rate increases if object weight increases or ground-robot friction increases. The mean and covariance calculations (IV-C) included all particles in the workspace. Particles that cannot reach the object due to obstacles skew these calculations. The state machine in Fig. 13a solves this problem by creating two states for the maze: either main or transfer. Each state has a set of regions representing a discretized visibility polygon. Whenever the object crosses a region boundary the state toggles. The main regions are generated by extending obstacles until they meet another obstacle. The transfer regions are perpendicular to obstacle boundaries, and act as a buffer between two main regions.

Fig. 13b shows the regions for the main state. The object is in region 1. An indicator function is applied to (IV-C) so only robots inside region 1 are counted. This filtering increases experimental success because the mean calculation only includes nearby robots that can directly interact with the object. When the object leaves main region 1 the state switches to transfer. The transfer regions are shown in Fig. 13c. The object is in transfer region 1, so only robots in transfer region 1 are included in the mean and covariance calculations. The robots should push the object to the left. Without filtering using regions, the red circle is the mean and the algorithm would instruct the robots to push the object up. The black circle shows the filtered mean and the algorithm instructs the robots to push the object directly left.

In 10 simulations with 100 robots, completion time without outlier rejection required 271$\pm$267 s while using outlier rejection required 90$\pm$35 s (mean$\pm$std).

VI-D Simulation results

We use the hybrid hysteresis-based controller in Alg. 1 to track the desired position, while maintaining sufficient robot density to move the object by switching to minimize variance whenever variance exceeds a set limit: $0.003W$ and $0.006W$ were added to the min and max variance limits from (IV-D). The minimize variance control law (20) is slightly modified to choose the nearest corner further from the goal than the object with an obstacle-free straight-line path to the object. The control algorithm for object manipulation is listed in Alg. 2.

In rare cases during simulations the swarm may become trapped in a local minimum of (34). If the swarm mean position does not change for five seconds, the swarm is assumed to be in a local minimum and is commanded to move toward the previous corner. As soon as the mean position changes, normal control resumes.

Fig. 15 shows snapshots during an execution of this algorithm in simulation. Experimental results of parameters sweeps are summarized in Fig. 16. Each trial measured the time to deliver the object to the goal location. The default parameter settings used 100 robots, a normalized weight of 1, a hexagon shape, and Brownian noise (applied once each simulation step) with $W=5$.

The interaction between the robots and object is impulsive so, like the study of impulsive pulling in [48], adding additional robots decreases completion time, but with diminishing returns. After 75 robots, additional robots no longer can interact with the object and do not contribute to the task success. Brownian noise adds stochasticity. This randomness can break the object free if it is stuck, but diminishes the effect of the control input. Increasing noise increases completion time. The robots have limited force, so increasing the object weight increases completion time. Each shape was designed to have the same mass and area. Rectangles and squares tend to get stuck in the 90∘ workspace corners, and cause longer completion times than circles, triangles, and hexagons.

VII Object manipulation with hardware robots

Our experiments use centimeter-scale hardware systems called kilobots. While those are far larger than the micro scale devices we model, using kilobots allows us to emulate a variety of dynamics, while enabling a high degree of control over robot function, the environment, and data collection. The kilobot, reported in [49, 50], is a low-cost robot designed for testing collective algorithms with large numbers of robots. It is available as an open-source platform or commercially from [51]. Each robot is approximately 3 cm in diameter, 3 cm tall, and uses two vibration motors to move on a flat surface at speeds up to 1 cm/s. Each robot has one ambient light sensor that is used to implement phototaxis, moving towards a light source.

VII-A Environmental setup

In these experiments as shown in Fig. 17, we used $n$=100 kilobots, a 1.5 m$\times$1.2 m whiteboard as the workspace, and lights: four 50W LED floodlights at the corners and four 30W LED floodlights on the sides of a 6 m square centered on the workspace and 1.5 m above the table. An Arduino Uno connected to an 8-relay shield controlled the lights.

Above the table, an overhead machine vision system tracks the swarm. The vision system identifies obstacles by color segmentation, determines the corners (used to decrease variance), the object by color segmentation, and identifies robots using color segmentation and circle detection with a circular Hough transform.

The objects were 3D printed from ABS plastic with a paper overlay. Shapes included a 325 cm2 equilateral triangle, 324 cm2 square, 281 cm2 hexagon, 254 cm2 circle, and a 486 cm2 (18 cm$\times$27 cm) rectangle, all shown in Fig. 17. The laser-cut patterns for the neon green fiducial markers on the robots and 3D files for objects are available at our github repository, [44].

Swarm mean control (hardware experiment)

Unlike the PD controller (11), we cannot command a force input to the kilobots. Instead, control is given by turning on one of eight lights. The kilobots run a phototaxis routine where they search for an orientation that aligns them with the light source, and then move with an approximately constant velocity toward this light. The kilobots oscillate along this orientation because they only have one light detector.

We use the sign of (11), and choose the closest orientation to $\mathbf{D}(\mathbf{b})$ among the eight light sources. Fig. 18 shows that this limited, discretized control still enables regulating the mean position of a swarm of 100 robots.

VII-B Automated object manipulation (hardware experiment)

The kilobots performed five successful runs manipulating a hexagonal object through an obstacle maze. Videos of these runs are in Extension 2. These hardware experiments represent the results of over 100 hours of trials. Each trial used 100 kilobots. Trials two through five were performed in a row with no failures in between. For each trial, fully charged kilobots were placed in the lower left-hand of the workspace, as shown in Fig. 19. The moveable object was placed in the lower center of the workspace. Matlab code for vision processing, the policy iteration of §VI-A and the algorithm of §VI-D is available on Matlab Central at [52]. Trials were run until the object COM entered the goal region. The trials ran for {1465, 3457, 3000, 2162, 2707} s. This is $2558\pm 771$ s (mean$\pm$std).

We also tested other object shapes. A circular object completed in 3155 s. A square object completed in 6871 s. A rectangle and three equilateral triangle objects of varying sizes failed in a total of nine runs. Manipulation failures occurred when the object was pushed into a corner, requiring torque to be unstuck. Swarm torque control is the subject of our ongoing research begun in [53].

VIII Conclusion

The small size of micro and nano particles makes individual control and autonomy challenging, so currently these particles are steered by global control inputs such as magnetic fields or chemical gradients. To investigate this control challenge, this paper introduced SwarmControl.net, an open-source tool for large-scale user experiments where human users steer swarms of robots to accomplish tasks. Analysis of the game play results revealed benefits of measuring and controlling statistics of the swarm rather than full state feedback, robustness to IID noise, and small effects of varying population size of large swarms.

Inspired by the three lessons from swarmcontrol.net, this paper designed controllers and controllability results using only the mean and variance of a particle swarm. We developed a hysteresis based controller to regulate the position and variance of a swarm. We designed a controller for object manipulation using policy iteration for path planning, regions for outlier rejection, and potential fields for minimizing moving the object backwards. All automatic controllers were implemented using 100 kilobots steered by the direction of a global light source. These experiments culminated in an object manipulation task in a workspace with obstacles.

Our future goal is to perform assembly using particle swarms to manipulate and attach components. This task requires controlling the position and orientation of components, manipulating them through obstacles and other components, and applying force and torque to components. This work provides foundational algorithms and techniques for steering swarms, object manipulation, and addressing obstacle fields, but there are many opportunities to extend the work.

Topics of interest include control with nonuniform flow such as fluid in an artery, gradient control fields like that of an MRI, competitive playing, multi-modal control, optimal-control, and targeted drug delivery in a vascular network.

Acknowledgements

We thank our anonymous reviewers and Dylan Shell for their substantive comments which improved the presentation and content of this paper.