# Deep Network Flow for Multi-Object Tracking

## TL;DR

This paper introduces a method to learn features for network-flow-based multi-object tracking by making the data association process differentiable, enabling end-to-end training and outperforming traditional hand-crafted costs.

## Contribution

It demonstrates that network flow costs in multi-object tracking can be learned via backpropagation, improving accuracy over hand-crafted methods.

## Key findings

- Learned cost functions outperform hand-crafted costs in all tested scenarios.
- End-to-end training simplifies the integration of multiple input sources.
- The approach is applicable to various data association problems in computer vision.

## Abstract

Data association problems are an important component of many computer vision applications, with multi-object tracking being one of the most prominent examples. A typical approach to data association involves finding a graph matching or network flow that minimizes a sum of pairwise association costs, which are often either hand-crafted or learned as linear functions of fixed features. In this work, we demonstrate that it is possible to learn features for network-flow-based data association via backpropagation, by expressing the optimum of a smoothed network flow problem as a differentiable function of the pairwise association costs. We apply this approach to multi-object tracking with a network flow formulation. Our experiments demonstrate that we are able to successfully learn all cost functions for the association problem in an end-to-end fashion, which outperform hand-crafted costs in all settings. The integration and combination of various sources of inputs becomes easy and the cost functions can be learned entirely from data, alleviating tedious hand-designing of costs.