Transforming Model Prediction for Tracking

TL;DR

This paper introduces a Transformer-based prediction module for tracking, enhancing global reasoning and model expressivity, leading to state-of-the-art performance on multiple benchmarks.

Contribution

It proposes a novel Transformer-based tracker architecture that learns more powerful target models and improves bounding box regression, trained end-to-end.

Findings

Achieves a 68.5% AUC on LaSOT dataset.

Sets new state-of-the-art results on three tracking benchmarks.

Demonstrates improved global reasoning in tracking models.

Abstract

Optimization based tracking methods have been widely successful by integrating a target model prediction module, providing effective global reasoning by minimizing an objective function. While this inductive bias integrates valuable domain knowledge, it limits the expressivity of the tracking network. In this work, we therefore propose a tracker architecture employing a Transformer-based model prediction module. Transformers capture global relations with little inductive bias, allowing it to learn the prediction of more powerful target models. We further extend the model predictor to estimate a second set of weights that are applied for accurate bounding box regression. The resulting tracker relies on training and on test frame information in order to predict all weights transductively. We train the proposed tracker end-to-end and validate its performance by conducting comprehensive experiments on multiple tracking datasets. Our tracker sets a new state of the art on three benchmarks, achieving an AUC of 68.5% on the challenging LaSOT dataset.