Audio Array-Based 3D UAV Trajectory Estimation with LiDAR Pseudo-Labeling

TL;DR

This paper presents a self-supervised framework for 3D UAV trajectory estimation using audio arrays and LiDAR pseudo-labeling, achieving high accuracy without requiring labeled data during deployment.

Contribution

It introduces a novel self-supervised learning approach that leverages LiDAR-based pseudo labels to train an audio-based UAV trajectory estimation model.

Findings

Achieves state-of-the-art performance on MMAUD dataset.

Eliminates the need for ground truth labels during inference.

Enhances tracking accuracy with Gaussian Process modeling.

Abstract

As small unmanned aerial vehicles (UAVs) become increasingly prevalent, there is growing concern regarding their impact on public safety and privacy, highlighting the need for advanced tracking and trajectory estimation solutions. In response, this paper introduces a novel framework that utilizes audio array for 3D UAV trajectory estimation. Our approach incorporates a self-supervised learning model, starting with the conversion of audio data into mel-spectrograms, which are analyzed through an encoder to extract crucial temporal and spectral information. Simultaneously, UAV trajectories are estimated using LiDAR point clouds via unsupervised methods. These LiDAR-based estimations act as pseudo labels, enabling the training of an Audio Perception Network without requiring labeled data. In this architecture, the LiDAR-based system operates as the Teacher Network, guiding the Audio Perception Network, which serves as the Student Network. Once trained, the model can independently predict 3D trajectories using only audio signals, with no need for LiDAR data or external ground truth during deployment. To further enhance precision, we apply Gaussian Process modeling for improved spatiotemporal tracking. Our method delivers top-tier performance on the MMAUD dataset, establishing a new benchmark in trajectory estimation using self-supervised learning techniques without reliance on ground truth annotations.