Unsupervised Domain Adaptation for Nighttime Aerial Tracking

TL;DR

This paper introduces a novel unsupervised domain adaptation framework for nighttime aerial tracking, utilizing a Transformer-based approach to align features and a new benchmark dataset to improve tracking performance in low-light conditions.

Contribution

It presents a unique object discovery method, a Transformer-based feature alignment, and a new benchmark dataset for unsupervised nighttime aerial tracking.

Findings

Demonstrates robustness of the proposed framework in nighttime conditions

Achieves effective domain adaptation from daytime to nighttime tracking

Provides a new benchmark dataset for future research

Abstract

Previous advances in object tracking mostly reported on favorable illumination circumstances while neglecting performance at nighttime, which significantly impeded the development of related aerial robot applications. This work instead develops a novel unsupervised domain adaptation framework for nighttime aerial tracking (named UDAT). Specifically, a unique object discovery approach is provided to generate training patches from raw nighttime tracking videos. To tackle the domain discrepancy, we employ a Transformer-based bridging layer post to the feature extractor to align image features from both domains. With a Transformer day/night feature discriminator, the daytime tracking model is adversarially trained to track at night. Moreover, we construct a pioneering benchmark namely NAT2021 for unsupervised domain adaptive nighttime tracking, which comprises a test set of 180 manually annotated tracking sequences and a train set of over 276k unlabelled nighttime tracking frames. Exhaustive experiments demonstrate the robustness and domain adaptability of the proposed framework in nighttime aerial tracking. The code and benchmark are available at https://github.com/vision4robotics/UDAT.