Real-Time Sense and Detect of Drones Using Deep Learning and Airborne LiDAR

TL;DR

This paper introduces an airborne LiDAR and deep learning-based system for real-time drone detection and localization, enhancing safety in drone swarms by providing a robust sense-and-detect capability beyond traditional methods.

Contribution

It presents the first LiDAR-based drone detection solution using 3D deep learning, with novel sparse convolution acceleration and synthetic data augmentation for close-proximity drone scenarios.

Findings

Achieved over 80% recall and 96% precision in real-world tests.

Developed a tracking-by-detection system for reliable drone separation monitoring.

Expanded neural network to handle air-to-air drone detection with improved speed.

Abstract

The safe operation of drone swarms beyond visual line of sight requires multiple safeguards to mitigate the risk of collision between drones flying in close-proximity scenarios. Cooperative navigation and flight coordination strategies that rely on pre-planned trajectories, constant %{satellite and network connectivity and reliable Global Navigation Satellite System (GNSS) positioning are brittle to failure. Drone embedded sense and detect offers a comprehensive mode of separation between drones for deconfliction and collision avoidance. This paper presents the first airborne LiDAR based solution for drone-swarm detection and localization using 3D deep learning model. It adapts an existing deep learning neural network to the air-to-air drone scenario by expanding the scan space vertically. A new sparse convolution is proposed and applied to accelerate the backbone layer, which is the most time-consuming part of the neural network. To collect training data of safety critical, close-proximity multi-drone operations, a scenario Digital Twin is used to augment real datasets with high fidelity synthetic data. The trained model achieves over 80% recall and 96% precision when tested on real-world datasets. By incorporating a tracking-by-detection algorithm the system can reliably monitor the separation distance of multiple drones in challenging environments.