Learning Coordinated Maneuver in Adversarial Environments

TL;DR

This paper develops a reinforcement learning-based approach for coordinating robot teams to minimize risk and time in adversarial environments, addressing complex non-convex problems with novel encoding and policies.

Contribution

It introduces a new RL framework with specialized encoding and policies for multi-robot coordination in adversarial settings, including theoretical analysis for single-adversary scenarios.

Findings

Reinforcement learning effectively produces coordinated team behaviors.

The approach reduces overall team cost in simulations.

Theoretical insights explain the behaviors leading to cost reduction.

Abstract

This paper aims to solve the coordination of a team of robots traversing a route in the presence of adversaries with random positions. Our goal is to minimize the overall cost of the team, which is determined by (i) the accumulated risk when robots stay in adversary-impacted zones and (ii) the mission completion time. During traversal, robots can reduce their speed and act as a `guard' (the slower, the better), which will decrease the risks certain adversary incurs. This leads to a trade-off between the robots' guarding behaviors and their travel speeds. The formulated problem is highly non-convex and cannot be efficiently solved by existing algorithms. Our approach includes a theoretical analysis of the robots' behaviors for the single-adversary case. As the scale of the problem expands, solving the optimal solution using optimization approaches is challenging, therefore, we employ reinforcement learning techniques by developing new encoding and policy-generating methods. Simulations demonstrate that our learning methods can efficiently produce team coordination behaviors. We discuss the reasoning behind these behaviors and explain why they reduce the overall team cost.