Correlation-Aware Deep Tracking

TL;DR

This paper introduces a correlation-aware deep tracking method that uses a novel target-dependent feature network with cross-image feature correlation, achieving state-of-the-art results in real-time tracking.

Contribution

The paper proposes a new target-dependent feature network with embedded cross-image correlation, improving robustness and discrimination in visual tracking.

Findings

Achieves state-of-the-art tracking performance.

Runs at real-time speed.

Can be integrated into existing tracking pipelines.

Abstract

Robustness and discrimination power are two fundamental requirements in visual object tracking. In most tracking paradigms, we find that the features extracted by the popular Siamese-like networks cannot fully discriminatively model the tracked targets and distractor objects, hindering them from simultaneously meeting these two requirements. While most methods focus on designing robust correlation operations, we propose a novel target-dependent feature network inspired by the self-/cross-attention scheme. In contrast to the Siamese-like feature extraction, our network deeply embeds cross-image feature correlation in multiple layers of the feature network. By extensively matching the features of the two images through multiple layers, it is able to suppress non-target features, resulting in instance-varying feature extraction. The output features of the search image can be directly used for predicting target locations without extra correlation step. Moreover, our model can be flexibly pre-trained on abundant unpaired images, leading to notably faster convergence than the existing methods. Extensive experiments show our method achieves the state-of-the-art results while running at real-time. Our feature networks also can be applied to existing tracking pipelines seamlessly to raise the tracking performance. Code will be available.