Learning a Neural Solver for Multiple Object Tracking

TL;DR

This paper introduces a fully differentiable neural network framework for multiple object tracking that operates directly on graph representations, enabling global reasoning and improving data association accuracy.

Contribution

It presents a novel Message Passing Network approach that integrates the data association step into the learning process for MOT, unlike previous methods.

Findings

Significant improvements in MOTA and IDF1 metrics.

Operates directly on graph domain for global reasoning.

Demonstrates effectiveness on three public benchmarks.

Abstract

Graphs offer a natural way to formulate Multiple Object Tracking (MOT) within the tracking-by-detection paradigm. However, they also introduce a major challenge for learning methods, as defining a model that can operate on such \textit{structured domain} is not trivial. As a consequence, most learning-based work has been devoted to learning better features for MOT, and then using these with well-established optimization frameworks. In this work, we exploit the classical network flow formulation of MOT to define a fully differentiable framework based on Message Passing Networks (MPNs). By operating directly on the graph domain, our method can reason globally over an entire set of detections and predict final solutions. Hence, we show that learning in MOT does not need to be restricted to feature extraction, but it can also be applied to the data association step. We show a significant improvement in both MOTA and IDF1 on three publicly available benchmarks. Our code is available at https://bit.ly/motsolv .