DCPT: Darkness Clue-Prompted Tracking in Nighttime UAVs

TL;DR

DCPT introduces a novel end-to-end trainable architecture for nighttime UAV tracking that directly encodes anti-dark features into prompts, outperforming traditional separate enhancement and tracking methods.

Contribution

The paper proposes Darkness Clue-Prompted Tracking (DCPT), a unified system that efficiently learns darkness cues and integrates them into a tracker without separate enhancement modules.

Findings

DCPT achieves state-of-the-art results on dark scenario benchmarks.

The darkness clue prompter effectively encodes anti-dark capabilities.

End-to-end training improves robustness and performance in nighttime UAV tracking.

Abstract

Existing nighttime unmanned aerial vehicle (UAV) trackers follow an "Enhance-then-Track" architecture - first using a light enhancer to brighten the nighttime video, then employing a daytime tracker to locate the object. This separate enhancement and tracking fails to build an end-to-end trainable vision system. To address this, we propose a novel architecture called Darkness Clue-Prompted Tracking (DCPT) that achieves robust UAV tracking at night by efficiently learning to generate darkness clue prompts. Without a separate enhancer, DCPT directly encodes anti-dark capabilities into prompts using a darkness clue prompter (DCP). Specifically, DCP iteratively learns emphasizing and undermining projections for darkness clues. It then injects these learned visual prompts into a daytime tracker with fixed parameters across transformer layers. Moreover, a gated feature aggregation mechanism enables adaptive fusion between prompts and between prompts and the base model. Extensive experiments show state-of-the-art performance for DCPT on multiple dark scenario benchmarks. The unified end-to-end learning of enhancement and tracking in DCPT enables a more trainable system. The darkness clue prompting efficiently injects anti-dark knowledge without extra modules. Code is available at https://github.com/bearyi26/DCPT.