Deformable Siamese Attention Networks for Visual Object Tracking

TL;DR

This paper introduces Deformable Siamese Attention Networks (SiamAttn) for visual object tracking, utilizing deformable attention mechanisms to improve template updating and contextual feature aggregation, leading to state-of-the-art results.

Contribution

The paper proposes a novel Siamese attention mechanism with deformable self- and cross-attention for adaptive template updating and enhanced context modeling in tracking.

Findings

Achieves new state-of-the-art results on six benchmarks.

Outperforms SiamRPN++ by significant margins on VOT 2016 and 2018.

Demonstrates effective adaptive template updating through cross-attention.

Abstract

Siamese-based trackers have achieved excellent performance on visual object tracking. However, the target template is not updated online, and the features of the target template and search image are computed independently in a Siamese architecture. In this paper, we propose Deformable Siamese Attention Networks, referred to as SiamAttn, by introducing a new Siamese attention mechanism that computes deformable self-attention and cross-attention. The self attention learns strong context information via spatial attention, and selectively emphasizes interdependent channel-wise features with channel attention. The cross-attention is capable of aggregating rich contextual inter-dependencies between the target template and the search image, providing an implicit manner to adaptively update the target template. In addition, we design a region refinement module that computes depth-wise cross correlations between the attentional features for more accurate tracking. We conduct experiments on six benchmarks, where our method achieves new state of-the-art results, outperforming the strong baseline, SiamRPN++ [24], by 0.464->0.537 and 0.415->0.470 EAO on VOT 2016 and 2018. Our code is available at: https://github.com/msight-tech/research-siamattn.