UETrack: A Unified and Efficient Framework for Single Object Tracking

TL;DR

UETrack is a versatile, efficient single object tracking framework capable of handling multiple modalities with a novel mixture-of-experts and adaptive distillation, achieving high accuracy and speed on various benchmarks.

Contribution

It introduces a unified framework that efficiently supports multi-modal inputs and employs innovative mechanisms for improved performance and resource efficiency.

Findings

Achieves 69.2% AUC on LaSOT benchmark.

Runs at 163/56/60 FPS on GPU/CPU/AGX hardware.

Outperforms previous methods in speed-accuracy trade-off.

Abstract

With growing real-world demands, efficient tracking has received increasing attention. However, most existing methods are limited to RGB inputs and struggle in multi-modal scenarios. Moreover, current multi-modal tracking approaches typically use complex designs, making them too heavy and slow for resource-constrained deployment. To tackle these limitations, we propose UETrack, an efficient framework for single object tracking. UETrack demonstrates high practicality and versatility, efficiently handling multiple modalities including RGB, Depth, Thermal, Event, and Language, and addresses the gap in efficient multi-modal tracking. It introduces two key components: a Token-Pooling-based Mixture-of-Experts mechanism that enhances modeling capacity through feature aggregation and expert specialization, and a Target-aware Adaptive Distillation strategy that selectively performs distillation based on sample characteristics, reducing redundant supervision and improving performance. Extensive experiments on 12 benchmarks across 3 hardware platforms show that UETrack achieves a superior speed-accuracy trade-off compared to previous methods. For instance, UETrack-B achieves 69.2% AUC on LaSOT and runs at 163/56/60 FPS on GPU/CPU/AGX, demonstrating strong practicality and versatility. Code is available at https://github.com/kangben258/UETrack.