Tracklet Association Tracker: An End-to-End Learning-based Association Approach for Multi-Object Tracking

TL;DR

This paper introduces the Tracklet Association Tracker (TAT), an end-to-end learning framework for multi-object tracking that integrates feature learning and data association through bi-level optimization, achieving state-of-the-art results.

Contribution

It presents a novel unified framework that directly learns association from features, significantly speeding up training and improving tracking performance.

Findings

Training is over 20 times faster than similar approaches.

Achieves state-of-the-art performance on MOT2016 and MOT2017 benchmarks.

Successfully integrates feature learning and data association in a single model.

Abstract

Traditional multiple object tracking methods divide the task into two parts: affinity learning and data association. The separation of the task requires to define a hand-crafted training goal in affinity learning stage and a hand-crafted cost function of data association stage, which prevents the tracking goals from learning directly from the feature. In this paper, we present a new multiple object tracking (MOT) framework with data-driven association method, named as Tracklet Association Tracker (TAT). The framework aims at gluing feature learning and data association into a unity by a bi-level optimization formulation so that the association results can be directly learned from features. To boost the performance, we also adopt the popular hierarchical association and perform the necessary alignment and selection of raw detection responses. Our model trains over 20X faster than a similar approach, and achieves the state-of-the-art performance on both MOT2016 and MOT2017 benchmarks.