A Discriminative Single-Shot Segmentation Network for Visual Object Tracking

TL;DR

D3S2 is a novel single-shot segmentation tracker that combines two target models with complementary properties to improve robustness and accuracy in visual object tracking and segmentation, outperforming many existing methods.

Contribution

It introduces a discriminative single-shot segmentation network that integrates invariant and rigid models for enhanced tracking without dataset-specific fine-tuning.

Findings

Outperforms all published trackers on VOT2020.

Performs close to state-of-the-art on GOT-10k, TrackingNet, OTB100, LaSoT.

Outperforms SiamMask on video object segmentation benchmarks.

Abstract

Template-based discriminative trackers are currently the dominant tracking paradigm due to their robustness, but are restricted to bounding box tracking and a limited range of transformation models, which reduces their localization accuracy. We propose a discriminative single-shot segmentation tracker -- D3S2, which narrows the gap between visual object tracking and video object segmentation. A single-shot network applies two target models with complementary geometric properties, one invariant to a broad range of transformations, including non-rigid deformations, the other assuming a rigid object to simultaneously achieve robust online target segmentation. The overall tracking reliability is further increased by decoupling the object and feature scale estimation. Without per-dataset finetuning, and trained only for segmentation as the primary output, D3S2 outperforms all published trackers on the recent short-term tracking benchmark VOT2020 and performs very close to the state-of-the-art trackers on the GOT-10k, TrackingNet, OTB100 and LaSoT. D3S2 outperforms the leading segmentation tracker SiamMask on video object segmentation benchmarks and performs on par with top video object segmentation algorithms.