Context-Aware Token Pruning and Discriminative Selective Attention for Transformer Tracking

TL;DR

This paper introduces CPDATrack, a novel transformer-based tracking framework that selectively prunes background tokens and employs discriminative attention to improve tracking accuracy and efficiency, especially in cluttered scenes.

Contribution

The paper proposes a learnable token filtering and a discriminative attention mechanism to enhance transformer tracking by reducing background interference and preserving contextual information.

Findings

Achieves state-of-the-art performance on GOT-10k with 75.1% average overlap.

Effectively suppresses distractors and background tokens, improving tracking accuracy.

Enhances computational efficiency through targeted token pruning.

Abstract

One-stream Transformer-based trackers have demonstrated remarkable performance by concatenating template and search region tokens, thereby enabling joint attention across all tokens. However, enabling an excessive proportion of background search tokens to attend to the target template tokens weakens the tracker's discriminative capability. Several token pruning methods have been proposed to mitigate background interference; however, they often remove tokens near the target, leading to the loss of essential contextual information and degraded tracking performance. Moreover, the presence of distractors within the search tokens further reduces the tracker's ability to accurately identify the target. To address these limitations, we propose CPDATrack, a novel tracking framework designed to suppress interference from background and distractor tokens while enhancing computational efficiency. First, a learnable module is integrated between two designated encoder layers to estimate the probability of each search token being associated with the target. Based on these estimates, less-informative background tokens are pruned from the search region while preserving the contextual cues surrounding the target. To further suppress background interference, a discriminative selective attention mechanism is employed that fully blocks search-to-template attention in the early layers. In the subsequent encoder layers, high-probability target tokens are selectively extracted from a localized region to attend to the template tokens, thereby reducing the influence of background and distractor tokens. The proposed CPDATrack achieves state-of-the-art performance across multiple benchmarks, particularly on GOT-10k, where it attains an average overlap of 75.1 percent.