Vision-based Drone Flocking in Outdoor Environments

TL;DR

This paper introduces a vision-based decentralized drone flocking system that uses neural networks for real-time detection and tracking without communication or markers, enabling safe outdoor navigation.

Contribution

It presents a novel neural network-based detection method combined with a multi-camera setup for autonomous outdoor drone flocking without external markers or communication.

Findings

Drones successfully navigate outdoors with real-time detection.

The system maintains safe flocking despite clutter and lighting challenges.

Open-source code and datasets support reproducibility.

Abstract

Decentralized deployment of drone swarms usually relies on inter-agent communication or visual markers that are mounted on the vehicles to simplify their mutual detection. This letter proposes a vision-based detection and tracking algorithm that enables groups of drones to navigate without communication or visual markers. We employ a convolutional neural network to detect and localize nearby agents onboard the quadcopters in real-time. Rather than manually labeling a dataset, we automatically annotate images to train the neural network using background subtraction by systematically flying a quadcopter in front of a static camera. We use a multi-agent state tracker to estimate the relative positions and velocities of nearby agents, which are subsequently fed to a flocking algorithm for high-level control. The drones are equipped with multiple cameras to provide omnidirectional visual inputs. The camera setup ensures the safety of the flock by avoiding blind spots regardless of the agent configuration. We evaluate the approach with a group of three real quadcopters that are controlled using the proposed vision-based flocking algorithm. The results show that the drones can safely navigate in an outdoor environment despite substantial background clutter and difficult lighting conditions. The source code, image dataset, and trained detection model are available at https://github.com/lis-epfl/vswarm.