MambaNUT: Nighttime UAV Tracking via Mamba-based Adaptive Curriculum Learning

TL;DR

This paper introduces MambaNUT, a novel lightweight nighttime UAV tracking framework utilizing a Mamba-based model and adaptive curriculum learning to enhance generalization and efficiency, achieving state-of-the-art results.

Contribution

The paper proposes a pure Mamba-based tracking framework with adaptive curriculum learning, reducing computational costs and improving nighttime UAV tracking performance.

Findings

Achieves state-of-the-art performance on nighttime UAV benchmarks.

Requires lower computational resources compared to ViT-based trackers.

Effectively balances daytime and nighttime data for robust tracking.

Abstract

Harnessing low-light enhancement and domain adaptation, nighttime UAV tracking has made substantial strides. However, over-reliance on image enhancement, limited high-quality nighttime data, and a lack of integration between daytime and nighttime trackers hinder the development of an end-to-end trainable framework. Additionally, current ViT-based trackers demand heavy computational resources due to their reliance on the self-attention mechanism. In this paper, we propose a novel pure Mamba-based tracking framework (MambaNUT) that employs a state space model with linear complexity as its backbone, incorporating a single-stream architecture that integrates feature learning and template-search coupling within Vision Mamba. We introduce an adaptive curriculum learning (ACL) approach that dynamically adjusts sampling strategies and loss weights, thereby improving the model's ability of generalization. Our ACL is composed of two levels of curriculum schedulers: (1) sampling scheduler that transforms the data distribution from imbalanced to balanced, as well as from easier (daytime) to harder (nighttime) samples; (2) loss scheduler that dynamically assigns weights based on the size of the training set and IoU of individual instances. Exhaustive experiments on multiple nighttime UAV tracking benchmarks demonstrate that the proposed MambaNUT achieves state-of-the-art performance while requiring lower computational costs. The code will be available at https://github.com/wuyou3474/MambaNUT.