RTAT: A Robust Two-stage Association Tracker for Multi-Object Tracking

TL;DR

RTAT introduces a two-stage association method for multi-object tracking that combines simple initial association with a graph neural network-based second stage, achieving state-of-the-art results on MOT benchmarks.

Contribution

The paper proposes a novel two-stage association framework, RTAT, that improves multi-object tracking by combining rule-based and learned association strategies with hierarchical graph modeling.

Findings

Achieves top performance on MOT17 and MOT20 benchmarks.

Outperforms existing methods in HOTA, IDF1, and AssA metrics.

Effectively merges short tracklets into longer trajectories.

Abstract

Data association is an essential part in the tracking-by-detection based Multi-Object Tracking (MOT). Most trackers focus on how to design a better data association strategy to improve the tracking performance. The rule-based handcrafted association methods are simple and highly efficient but lack generalization capability to deal with complex scenes. While the learnt association methods can learn high-order contextual information to deal with various complex scenes, but they have the limitations of higher complexity and cost. To address these limitations, we propose a Robust Two-stage Association Tracker, named RTAT. The first-stage association is performed between tracklets and detections to generate tracklets with high purity, and the second-stage association is performed between tracklets to form complete trajectories. For the first-stage association, we use a simple data association strategy to generate tracklets with high purity by setting a low threshold for the matching cost in the assignment process. We conduct the tracklet association in the second-stage based on the framework of message-passing GNN. Our method models the tracklet association as a series of edge classification problem in hierarchical graphs, which can recursively merge short tracklets into longer ones. Our tracker RTAT ranks first on the test set of MOT17 and MOT20 benchmarks in most of the main MOT metrics: HOTA, IDF1, and AssA. We achieve 67.2 HOTA, 84.7 IDF1, and 69.7 AssA on MOT17, and 66.2 HOTA, 82.5 IDF1, and 68.1 AssA on MOT20.