Ultra-High-Frequency Harmony: mmWave Radar and Event Camera Orchestrate Accurate Drone Landing

TL;DR

This paper introduces mmE-Loc, a novel high-precision drone localization system that combines mmWave radar and event cameras with innovative fusion modules, achieving superior accuracy and low latency for drone landing tasks.

Contribution

The work presents a new sensor fusion framework using event cameras and mmWave radar, with novel modules for improved drone localization in real-time landing scenarios.

Findings

Outperforms state-of-the-art localization methods in accuracy.

Reduces latency in drone landing localization.

Demonstrated effectiveness in real-world drone delivery scenarios.

Abstract

For precise, efficient, and safe drone landings, ground platforms should real-time, accurately locate descending drones and guide them to designated spots. While mmWave sensing combined with cameras improves localization accuracy, the lower sampling frequency of traditional frame cameras compared to mmWave radar creates bottlenecks in system throughput. In this work, we replace the traditional frame camera with event camera, a novel sensor that harmonizes in sampling frequency with mmWave radar within the ground platform setup, and introduce mmE-Loc, a high-precision, low-latency ground localization system designed for drone landings. To fully leverage the \textit{temporal consistency} and \textit{spatial complementarity} between these modalities, we propose two innovative modules, \textit{consistency-instructed collaborative tracking} and \textit{graph-informed adaptive joint optimization}, for accurate drone measurement extraction and efficient sensor fusion. Extensive real-world experiments in landing scenarios from a leading drone delivery company demonstrate that mmE-Loc outperforms state-of-the-art methods in both localization accuracy and latency.