Less is More: Token Context-aware Learning for Object Tracking

TL;DR

This paper introduces LMTrack, a novel object tracking method that selectively learns high-quality reference tokens using a token memory and attention mechanism, leading to improved accuracy and robustness in visual tracking.

Contribution

The paper proposes a token context-aware tracking pipeline with a new memory module and attention mechanism to automatically select relevant tokens, reducing noise and redundancy.

Findings

Achieves state-of-the-art results on GOT-10K, TrackingNet, and LaSOT.

Effectively filters background noise to improve tracking accuracy.

Demonstrates robustness across various challenging tracking scenarios.

Abstract

Recently, several studies have shown that utilizing contextual information to perceive target states is crucial for object tracking. They typically capture context by incorporating multiple video frames. However, these naive frame-context methods fail to consider the importance of each patch within a reference frame, making them susceptible to noise and redundant tokens, which deteriorates tracking performance. To address this challenge, we propose a new token context-aware tracking pipeline named LMTrack, designed to automatically learn high-quality reference tokens for efficient visual tracking. Embracing the principle of Less is More, the core idea of LMTrack is to analyze the importance distribution of all reference tokens, where important tokens are collected, continually attended to, and updated. Specifically, a novel Token Context Memory module is designed to dynamically collect high-quality spatio-temporal information of a target in an autoregressive manner, eliminating redundant background tokens from the reference frames. Furthermore, an effective Unidirectional Token Attention mechanism is designed to establish dependencies between reference tokens and search frame, enabling robust cross-frame association and target localization. Extensive experiments demonstrate the superiority of our tracker, achieving state-of-the-art results on tracking benchmarks such as GOT-10K, TrackingNet, and LaSOT.