Nearest-Neighbor-based Collision Avoidance for Quadrotors via Reinforcement Learning

TL;DR

This paper introduces a decentralized, reinforcement learning-based collision avoidance method for quadrotors inspired by starling flocking behavior, demonstrating effective real-world application with a scalable, biomimetic observation model.

Contribution

It proposes a novel, scalable nearest-neighbor observation model combined with reinforcement learning for decentralized collision avoidance in quadrotors, applicable to complex motion models and real-world scenarios.

Findings

Effective collision avoidance learned with nearest-neighbor info

Successful transfer of policies from simulation to real drones

Applicable to various tasks like package collection and formation change

Abstract

Collision avoidance algorithms are of central interest to many drone applications. In particular, decentralized approaches may be the key to enabling robust drone swarm solutions in cases where centralized communication becomes computationally prohibitive. In this work, we draw biological inspiration from flocks of starlings (Sturnus vulgaris) and apply the insight to end-to-end learned decentralized collision avoidance. More specifically, we propose a new, scalable observation model following a biomimetic nearest-neighbor information constraint that leads to fast learning and good collision avoidance behavior. By proposing a general reinforcement learning approach, we obtain an end-to-end learning-based approach to integrating collision avoidance with arbitrary tasks such as package collection and formation change. To validate the generality of this approach, we successfully apply our methodology through motion models of medium complexity, modeling momentum and nonetheless allowing direct application to real world quadrotors in conjunction with a standard PID controller. In contrast to prior works, we find that in our sufficiently rich motion model, nearest-neighbor information is indeed enough to learn effective collision avoidance behavior. Our learned policies are tested in simulation and subsequently transferred to real-world drones to validate their real-world applicability.