A Twofold Siamese Network for Real-Time Object Tracking

TL;DR

This paper introduces SA-Siam, a real-time object tracking network combining semantic and appearance features through separate training and a channel attention mechanism, significantly enhancing tracking accuracy.

Contribution

The paper proposes a novel twofold Siamese network with separate training for semantic and appearance branches, incorporating a channel attention mechanism for improved real-time tracking.

Findings

Outperforms all other real-time trackers on OTB benchmarks

Significantly improves tracking performance with the twofold design

Channel attention mechanism enhances feature relevance

Abstract

Observing that Semantic features learned in an image classification task and Appearance features learned in a similarity matching task complement each other, we build a twofold Siamese network, named SA-Siam, for real-time object tracking. SA-Siam is composed of a semantic branch and an appearance branch. Each branch is a similarity-learning Siamese network. An important design choice in SA-Siam is to separately train the two branches to keep the heterogeneity of the two types of features. In addition, we propose a channel attention mechanism for the semantic branch. Channel-wise weights are computed according to the channel activations around the target position. While the inherited architecture from SiamFC \cite{SiamFC} allows our tracker to operate beyond real-time, the twofold design and the attention mechanism significantly improve the tracking performance. The proposed SA-Siam outperforms all other real-time trackers by a large margin on OTB-2013/50/100 benchmarks.