Multi-State Tracker: Enhancing Efficient Object Tracking via Multi-State Specialization and Interaction

TL;DR

The paper introduces Multi-State Tracker (MST), a lightweight and efficient object tracking method that enhances feature representation through multi-state specialization and interaction, leading to improved accuracy and robustness with minimal computational cost.

Contribution

MST employs lightweight multi-state feature enhancement and interaction modules, significantly improving tracking performance while maintaining high efficiency compared to prior methods.

Findings

Outperforms previous efficient trackers in accuracy and robustness.

Achieves a 4.5% AO score improvement on GOT-10K dataset.

Uses only 0.1 GFLOPs and 0.66 M parameters, demonstrating high efficiency.

Abstract

Efficient trackers achieve faster runtime by reducing computational complexity and model parameters. However, this efficiency often compromises the expense of weakened feature representation capacity, thus limiting their ability to accurately capture target states using single-layer features. To overcome this limitation, we propose Multi-State Tracker (MST), which utilizes highly lightweight state-specific enhancement (SSE) to perform specialized enhancement on multi-state features produced by multi-state generation (MSG) and aggregates them in an interactive and adaptive manner using cross-state interaction (CSI). This design greatly enhances feature representation while incurring minimal computational overhead, leading to improved tracking robustness in complex environments. Specifically, the MSG generates multiple state representations at multiple stages during feature extraction, while SSE refines them to highlight target-specific features. The CSI module facilitates information exchange between these states and ensures the integration of complementary features. Notably, the introduced SSE and CSI modules adopt a highly lightweight hidden state adaptation-based state space duality (HSA-SSD) design, incurring only 0.1 GFLOPs in computation and 0.66 M in parameters. Experimental results demonstrate that MST outperforms all previous efficient trackers across multiple datasets, significantly improving tracking accuracy and robustness. In particular, it shows excellent runtime performance, with an AO score improvement of 4.5\% over the previous SOTA efficient tracker HCAT on the GOT-10K dataset. The code is available at https://github.com/wsumel/MST.