TSDM: Tracking by SiamRPN++ with a Depth-refiner and a Mask-generator

TL;DR

This paper introduces TSDM, a novel RGB-D object tracker that leverages depth information through a Mask-generator and Depth-refiner to improve tracking accuracy and robustness, outperforming state-of-the-art methods.

Contribution

The paper presents a new RGB-D tracker with a depth-refiner and mask-generator, effectively utilizing depth cues for enhanced object tracking performance.

Findings

Outperforms state-of-the-art trackers on Princeton and VOT benchmarks.

Achieves 23 FPS with the full model and 31 FPS with a lightweight variant.

Demonstrates practical real-world application potential.

Abstract

In a generic object tracking, depth (D) information provides informative cues for foreground-background separation and target bounding box regression. However, so far, few trackers have used depth information to play the important role aforementioned due to the lack of a suitable model. In this paper, a RGB-D tracker named TSDM is proposed, which is composed of a Mask-generator (M-g), SiamRPN++ and a Depth-refiner (D-r). The M-g generates the background masks, and updates them as the target 3D position changes. The D-r optimizes the target bounding box estimated by SiamRPN++, based on the spatial depth distribution difference between the target and the surrounding background. Extensive evaluation on the Princeton Tracking Benchmark and the Visual Object Tracking challenge shows that our tracker outperforms the state-of-the-art by a large margin while achieving 23 FPS. In addition, a light-weight variant can run at 31 FPS and thus it is practical for real world applications. Code and models of TSDM are available at https://github.com/lql-team/TSDM.