3D UAV Trajectory Estimation and Classification from Internet Videos via Language Model

TL;DR

This paper introduces a novel framework for estimating and classifying 3D UAV trajectories directly from Internet videos using language models, eliminating the need for manual annotations and enabling zero-shot transfer to benchmarks.

Contribution

The work presents a new, annotation-free approach combining language-driven data collection, cross-modal label generation, and physics-informed refinement for UAV trajectory estimation from online videos.

Findings

Performance improves with more online video data.

Method approaches state-of-the-art accuracy.

Effective in zero-shot transfer without target domain training.

Abstract

Reliable 3D trajectory estimation of unmanned aerial vehicles (UAVs) is a fundamental requirement for anti-UAV systems, yet the acquisition of large-scale and accurately annotated trajectory data remains prohibitively expensive. In this work, we present a novel framework that derives UAV 3D trajectories and category information directly from Internet-scale UAV videos, without relying on manual annotations. First, language-driven data acquisition is employed to autonomously discover and collect UAV-related videos, while vision-language reasoning progressively filters task-relevant segments. Second, a training-free cross-modal label generation module is introduced to infer 3D trajectory hypotheses and UAV type cues. Third, a physics-informed refinement process is designed to impose temporal smoothness and kinematic consistency on the estimated trajectories. The resulting video clips and trajectory annotations can be readily utilized for downstream anti-UAV tasks. To assess effectiveness and generalization, we conduct zero-shot transfer experiments on a public, well-annotated 3D UAV benchmark. Results reveal a clear data scaling behavior: as the amount of online video data increases, zero-shot transfer performance on the target dataset improves consistently, without any target-domain training. The proposed method closely approaches the current state-of-the-art, highlighting its robustness and applicability to real-world anti-UAV scenarios. Code and datasets will be released upon acceptance.