Vision-based Relative Detection and Tracking for Teams of Micro Aerial Vehicles

TL;DR

This paper presents a vision-based decentralized multi-tracking system for teams of micro aerial vehicles, utilizing Bayesian filtering and deep learning to achieve accurate, real-time detection and tracking under challenging conditions.

Contribution

It introduces novel Bayesian multi-tracking filters combined with a lightweight DNN detector for decentralized, real-time multi-drone tracking on SWaP-constrained robots.

Findings

Effective tracking of multiple drones in occlusion-heavy scenarios

Comparison of filtering strategies for accuracy and scalability

Demonstration of real-time operation on SWaP-constrained hardware

Abstract

In this paper, we address the vision-based detection and tracking problems of multiple aerial vehicles using a single camera and Inertial Measurement Unit (IMU) as well as the corresponding perception consensus problem (i.e., uniqueness and identical IDs across all observing agents). We design several vision-based decentralized Bayesian multi-tracking filtering strategies to resolve the association between the incoming unsorted measurements obtained by a visual detector algorithm and the tracked agents. We compare their accuracy in different operating conditions as well as their scalability according to the number of agents in the team. This analysis provides useful insights about the most appropriate design choice for the given task. We further show that the proposed perception and inference pipeline which includes a Deep Neural Network (DNN) as visual target detector is lightweight and capable of concurrently running control and planning with Size, Weight, and Power (SWaP) constrained robots on-board. Experimental results show the effective tracking of multiple drones in various challenging scenarios such as heavy occlusions.