Deeper and Wider Siamese Networks for Real-Time Visual Tracking

TL;DR

This paper introduces deeper and wider Siamese network architectures with residual modules to improve real-time visual tracking accuracy and robustness, addressing issues caused by increased receptive fields and padding effects.

Contribution

The authors propose new residual modules and architectures that control receptive field size and mitigate padding bias, enhancing Siamese network performance in visual tracking.

Findings

Up to 9.8% improvement in AUC on OTB-15

Up to 23.3% improvement in EAO on VOT-16

Achieves real-time tracking speed with enhanced accuracy

Abstract

Siamese networks have drawn great attention in visual tracking because of their balanced accuracy and speed. However, the backbone networks used in Siamese trackers are relatively shallow, such as AlexNet [18], which does not fully take advantage of the capability of modern deep neural networks. In this paper, we investigate how to leverage deeper and wider convolutional neural networks to enhance tracking robustness and accuracy. We observe that direct replacement of backbones with existing powerful architectures, such as ResNet [14] and Inception [33], does not bring improvements. The main reasons are that 1)large increases in the receptive field of neurons lead to reduced feature discriminability and localization precision; and 2) the network padding for convolutions induces a positional bias in learning. To address these issues, we propose new residual modules to eliminate the negative impact of padding, and further design new architectures using these modules with controlled receptive field size and network stride. The designed architectures are lightweight and guarantee real-time tracking speed when applied to SiamFC [2] and SiamRPN [20]. Experiments show that solely due to the proposed network architectures, our SiamFC+ and SiamRPN+ obtain up to 9.8%/5.7% (AUC), 23.3%/8.8% (EAO) and 24.4%/25.0% (EAO) relative improvements over the original versions [2, 20] on the OTB-15, VOT-16 and VOT-17 datasets, respectively.