Scale Equivariance Improves Siamese Tracking

TL;DR

This paper introduces a scale-equivariant extension to Siamese trackers, improving their ability to handle object scale variations by incorporating built-in scale equivariance, validated through experiments on multiple benchmarks.

Contribution

The paper develops the theory for scale-equivariant Siamese trackers and provides a practical method to enhance existing trackers with scale equivariance.

Findings

Scale-equivariant Siamese tracker improves tracking accuracy.

Built-in scale equivariance enhances robustness to size changes.

Experimental results show superior performance on benchmarks.

Abstract

Siamese trackers turn tracking into similarity estimation between a template and the candidate regions in the frame. Mathematically, one of the key ingredients of success of the similarity function is translation equivariance. Non-translation-equivariant architectures induce a positional bias during training, so the location of the target will be hard to recover from the feature space. In real life scenarios, objects undergoe various transformations other than translation, such as rotation or scaling. Unless the model has an internal mechanism to handle them, the similarity may degrade. In this paper, we focus on scaling and we aim to equip the Siamese network with additional built-in scale equivariance to capture the natural variations of the target a priori. We develop the theory for scale-equivariant Siamese trackers, and provide a simple recipe for how to make a wide range of existing trackers scale-equivariant. We present SE-SiamFC, a scale-equivariant variant of SiamFC built according to the recipe. We conduct experiments on OTB and VOT benchmarks and on the synthetically generated T-MNIST and S-MNIST datasets. We demonstrate that a built-in additional scale equivariance is useful for visual object tracking.