Unsupervised Learning of Accurate Siamese Tracking

TL;DR

This paper introduces a novel unsupervised Siamese tracking framework that leverages cycle consistency and dynamic loss reweighting to improve long-term object tracking without supervision, achieving results comparable to supervised methods.

Contribution

It proposes a new unsupervised tracking method using cycle consistency, a differentiable region mask, and dynamic loss reweighting to enhance accuracy and robustness.

Findings

Outperforms previous unsupervised methods significantly.

Achieves comparable performance to supervised trackers on large datasets.

Demonstrates robustness in long-term tracking scenarios.

Abstract

Unsupervised learning has been popular in various computer vision tasks, including visual object tracking. However, prior unsupervised tracking approaches rely heavily on spatial supervision from template-search pairs and are still unable to track objects with strong variation over a long time span. As unlimited self-supervision signals can be obtained by tracking a video along a cycle in time, we investigate evolving a Siamese tracker by tracking videos forward-backward. We present a novel unsupervised tracking framework, in which we can learn temporal correspondence both on the classification branch and regression branch. Specifically, to propagate reliable template feature in the forward propagation process so that the tracker can be trained in the cycle, we first propose a consistency propagation transformation. We then identify an ill-posed penalty problem in conventional cycle training in backward propagation process. Thus, a differentiable region mask is proposed to select features as well as to implicitly penalize tracking errors on intermediate frames. Moreover, since noisy labels may degrade training, we propose a mask-guided loss reweighting strategy to assign dynamic weights based on the quality of pseudo labels. In extensive experiments, our tracker outperforms preceding unsupervised methods by a substantial margin, performing on par with supervised methods on large-scale datasets such as TrackingNet and LaSOT. Code is available at https://github.com/FlorinShum/ULAST.