Quo Vadis: Is Trajectory Forecasting the Key Towards Long-Term Multi-Object Tracking?

TL;DR

This paper proposes that trajectory forecasting over longer time horizons can significantly enhance long-term multi-object tracking by reducing the search space for object associations during occlusions.

Contribution

It introduces a method that uses diverse trajectory predictions in bird's-eye view to improve long-term tracking robustness, advancing state-of-the-art performance.

Findings

Significant improvement in long-term tracking on MOTChallenge dataset.

Diverse trajectory forecasts reduce association search space.

Reasoning in bird's-eye view enhances prediction accuracy.

Abstract

Recent developments in monocular multi-object tracking have been very successful in tracking visible objects and bridging short occlusion gaps, mainly relying on data-driven appearance models. While we have significantly advanced short-term tracking performance, bridging longer occlusion gaps remains elusive: state-of-the-art object trackers only bridge less than 10% of occlusions longer than three seconds. We suggest that the missing key is reasoning about future trajectories over a longer time horizon. Intuitively, the longer the occlusion gap, the larger the search space for possible associations. In this paper, we show that even a small yet diverse set of trajectory predictions for moving agents will significantly reduce this search space and thus improve long-term tracking robustness. Our experiments suggest that the crucial components of our approach are reasoning in a bird's-eye view space and generating a small yet diverse set of forecasts while accounting for their localization uncertainty. This way, we can advance state-of-the-art trackers on the MOTChallenge dataset and significantly improve their long-term tracking performance. This paper's source code and experimental data are available at https://github.com/dendorferpatrick/QuoVadis.