Learning Feature Embeddings for Discriminant Model based Tracking

TL;DR

This paper introduces DCFST, a novel neural network architecture that learns optimal feature embeddings for discriminant model-based online tracking, achieving state-of-the-art accuracy and real-time performance across multiple benchmarks.

Contribution

It proposes an end-to-end trainable framework integrating a differentiable discriminant solver into CNNs for improved online tracking.

Findings

Achieves state-of-the-art accuracy on six benchmarks.

Runs faster than real-time in online tracking.

Generalizes well to class-agnostic targets.

Abstract

After observing that the features used in most online discriminatively trained trackers are not optimal, in this paper, we propose a novel and effective architecture to learn optimal feature embeddings for online discriminative tracking. Our method, called DCFST, integrates the solver of a discriminant model that is differentiable and has a closed-form solution into convolutional neural networks. Then, the resulting network can be trained in an end-to-end way, obtaining optimal feature embeddings for the discriminant model-based tracker. As an instance, we apply the popular ridge regression model in this work to demonstrate the power of DCFST. Extensive experiments on six public benchmarks, OTB2015, NFS, GOT10k, TrackingNet, VOT2018, and VOT2019, show that our approach is efficient and generalizes well to class-agnostic target objects in online tracking, thus achieves state-of-the-art accuracy, while running beyond the real-time speed. Code will be made available.