Autoregressive Queries for Adaptive Tracking with Spatio-TemporalTransformers

TL;DR

This paper introduces AQATrack, an adaptive visual tracking method using autoregressive queries and spatio-temporal transformers to effectively capture target appearance variations without extensive hand-crafted components.

Contribution

The paper proposes a novel autoregressive query mechanism and a spatio-temporal fusion module for improved adaptive tracking with fewer manual design elements.

Findings

Significantly outperforms existing methods on six tracking benchmarks.

Effectively captures instantaneous target appearance changes.

Enhances robustness by combining static and dynamic information.

Abstract

The rich spatio-temporal information is crucial to capture the complicated target appearance variations in visual tracking. However, most top-performing tracking algorithms rely on many hand-crafted components for spatio-temporal information aggregation. Consequently, the spatio-temporal information is far away from being fully explored. To alleviate this issue, we propose an adaptive tracker with spatio-temporal transformers (named AQATrack), which adopts simple autoregressive queries to effectively learn spatio-temporal information without many hand-designed components. Firstly, we introduce a set of learnable and autoregressive queries to capture the instantaneous target appearance changes in a sliding window fashion. Then, we design a novel attention mechanism for the interaction of existing queries to generate a new query in current frame. Finally, based on the initial target template and learnt autoregressive queries, a spatio-temporal information fusion module (STM) is designed for spatiotemporal formation aggregation to locate a target object. Benefiting from the STM, we can effectively combine the static appearance and instantaneous changes to guide robust tracking. Extensive experiments show that our method significantly improves the tracker's performance on six popular tracking benchmarks: LaSOT, LaSOText, TrackingNet, GOT-10k, TNL2K, and UAV123.