Collision Avoidance and Navigation for a Quadrotor Swarm Using End-to-end Deep Reinforcement Learning

TL;DR

This paper introduces an end-to-end deep reinforcement learning method for quadrotor swarms that effectively navigates obstacle-rich environments, demonstrating zero-shot transfer from simulation to real-world hardware with scalable multi-robot control.

Contribution

It presents the first successful use of attention mechanisms in DRL for swarm navigation, enabling obstacle and neighbor avoidance on hardware-constrained quadrotors.

Findings

Scales to 32 robots in simulation with 80% obstacle density

Successfully transfers policies to 8 real quadrotors in physical environments

Achieves obstacle and neighbor avoidance with end-to-end DRL

Abstract

End-to-end deep reinforcement learning (DRL) for quadrotor control promises many benefits -- easy deployment, task generalization and real-time execution capability. Prior end-to-end DRL-based methods have showcased the ability to deploy learned controllers onto single quadrotors or quadrotor teams maneuvering in simple, obstacle-free environments. However, the addition of obstacles increases the number of possible interactions exponentially, thereby increasing the difficulty of training RL policies. In this work, we propose an end-to-end DRL approach to control quadrotor swarms in environments with obstacles. We provide our agents a curriculum and a replay buffer of the clipped collision episodes to improve performance in obstacle-rich environments. We implement an attention mechanism to attend to the neighbor robots and obstacle interactions - the first successful demonstration of this mechanism on policies for swarm behavior deployed on severely compute-constrained hardware. Our work is the first work that demonstrates the possibility of learning neighbor-avoiding and obstacle-avoiding control policies trained with end-to-end DRL that transfers zero-shot to real quadrotors. Our approach scales to 32 robots with 80% obstacle density in simulation and 8 robots with 20% obstacle density in physical deployment. Video demonstrations are available on the project website at: https://sites.google.com/view/obst-avoid-swarm-rl.