ACTrack: Adding Spatio-Temporal Condition for Visual Object Tracking

TL;DR

ACTrack introduces a lightweight additive spatio-temporal model for visual object tracking that preserves pre-trained backbone capabilities while improving efficiency and performance.

Contribution

It proposes a novel additive spatio-temporal framework with a lightweight net that maintains pre-trained model quality and enhances tracking efficiency.

Findings

Balances training efficiency and tracking performance.

Outperforms existing methods on multiple benchmarks.

Preserves pre-trained Transformer backbone capabilities.

Abstract

Efficiently modeling spatio-temporal relations of objects is a key challenge in visual object tracking (VOT). Existing methods track by appearance-based similarity or long-term relation modeling, resulting in rich temporal contexts between consecutive frames being easily overlooked. Moreover, training trackers from scratch or fine-tuning large pre-trained models needs more time and memory consumption. In this paper, we present ACTrack, a new tracking framework with additive spatio-temporal conditions. It preserves the quality and capabilities of the pre-trained Transformer backbone by freezing its parameters, and makes a trainable lightweight additive net to model spatio-temporal relations in tracking. We design an additive siamese convolutional network to ensure the integrity of spatial features and perform temporal sequence modeling to simplify the tracking pipeline. Experimental results on several benchmarks prove that ACTrack could balance training efficiency and tracking performance.