SAM-DA: UAV Tracks Anything at Night with SAM-Powered Domain Adaptation

TL;DR

SAM-DA introduces a novel domain adaptation framework powered by the Segment Anything Model to enhance real-time nighttime UAV tracking, significantly expanding training samples and improving performance with fewer images.

Contribution

The paper proposes a SAM-powered domain adaptation method that generates high-quality training samples from single nighttime images, enabling more effective and efficient UAV tracking in challenging conditions.

Findings

SAM-DA outperforms state-of-the-art domain adaptation methods.

Fewer training images are needed for effective nighttime UAV tracking.

The approach demonstrates robustness and quick deployment capabilities.

Abstract

Domain adaptation (DA) has demonstrated significant promise for real-time nighttime unmanned aerial vehicle (UAV) tracking. However, the state-of-the-art (SOTA) DA still lacks the potential object with accurate pixel-level location and boundary to generate the high-quality target domain training sample. This key issue constrains the transfer learning of the real-time daytime SOTA trackers for challenging nighttime UAV tracking. Recently, the notable Segment Anything Model (SAM) has achieved a remarkable zero-shot generalization ability to discover abundant potential objects due to its huge data-driven training approach. To solve the aforementioned issue, this work proposes a novel SAM-powered DA framework for real-time nighttime UAV tracking, i.e., SAM-DA. Specifically, an innovative SAM-powered target domain training sample swelling is designed to determine enormous high-quality target domain training samples from every single raw nighttime image. This novel one-to-many generation significantly expands the high-quality target domain training sample for DA. Comprehensive experiments on extensive nighttime UAV videos prove the robustness and domain adaptability of SAM-DA for nighttime UAV tracking. Especially, compared to the SOTA DA, SAM-DA can achieve better performance with fewer raw nighttime images, i.e., the fewer-better training. This economized training approach facilitates the quick validation and deployment of algorithms for UAVs. The code is available at https://github.com/vision4robotics/SAM-DA.