TrackFlow: Multi-Object Tracking with Normalizing Flows

TL;DR

TrackFlow introduces a probabilistic approach using normalizing flows to improve multi-object tracking by modeling the joint probability of associations, effectively integrating multi-modal cues and outperforming existing methods.

Contribution

This paper presents a novel probabilistic framework with normalizing flows for multi-object tracking, enabling better integration of heterogeneous data without extensive hyperparameter tuning.

Findings

Consistently improves tracking accuracy across benchmarks

Effectively combines multi-modal cues in tracking

Outperforms traditional heuristic-based methods

Abstract

The field of multi-object tracking has recently seen a renewed interest in the good old schema of tracking-by-detection, as its simplicity and strong priors spare it from the complex design and painful babysitting of tracking-by-attention approaches. In view of this, we aim at extending tracking-by-detection to multi-modal settings, where a comprehensive cost has to be computed from heterogeneous information e.g., 2D motion cues, visual appearance, and pose estimates. More precisely, we follow a case study where a rough estimate of 3D information is also available and must be merged with other traditional metrics (e.g., the IoU). To achieve that, recent approaches resort to either simple rules or complex heuristics to balance the contribution of each cost. However, i) they require careful tuning of tailored hyperparameters on a hold-out set, and ii) they imply these costs to be independent, which does not hold in reality. We address these issues by building upon an elegant probabilistic formulation, which considers the cost of a candidate association as the negative log-likelihood yielded by a deep density estimator, trained to model the conditional joint probability distribution of correct associations. Our experiments, conducted on both simulated and real benchmarks, show that our approach consistently enhances the performance of several tracking-by-detection algorithms.