EVPropNet: Detecting Drones By Finding Propellers For Mid-Air Landing And Following

TL;DR

This paper introduces EVPropNet, a deep learning-based system utilizing event cameras to detect drone propellers for tracking and landing, achieving high accuracy and real-world applicability without retraining.

Contribution

The paper presents the first deep learning approach for propeller detection using event cameras, enabling drone tracking and landing without fine-tuning.

Findings

Detects propellers with 85.1% accuracy even with 60% occlusion.

Operates at up to 35Hz with 2W power consumption.

Achieves 92% success in tracking and 90% in landing tasks.

Abstract

The rapid rise of accessibility of unmanned aerial vehicles or drones pose a threat to general security and confidentiality. Most of the commercially available or custom-built drones are multi-rotors and are comprised of multiple propellers. Since these propellers rotate at a high-speed, they are generally the fastest moving parts of an image and cannot be directly "seen" by a classical camera without severe motion blur. We utilize a class of sensors that are particularly suitable for such scenarios called event cameras, which have a high temporal resolution, low-latency, and high dynamic range. In this paper, we model the geometry of a propeller and use it to generate simulated events which are used to train a deep neural network called EVPropNet to detect propellers from the data of an event camera. EVPropNet directly transfers to the real world without any fine-tuning or retraining. We present two applications of our network: (a) tracking and following an unmarked drone and (b) landing on a near-hover drone. We successfully evaluate and demonstrate the proposed approach in many real-world experiments with different propeller shapes and sizes. Our network can detect propellers at a rate of 85.1% even when 60% of the propeller is occluded and can run at upto 35Hz on a 2W power budget. To our knowledge, this is the first deep learning-based solution for detecting propellers (to detect drones). Finally, our applications also show an impressive success rate of 92% and 90% for the tracking and landing tasks respectively.