Improving tracking with a tracklet associator

TL;DR

This paper introduces a modular Constraint Programming-based approach to enhance multiple object tracking by improving object association accuracy, leading to significant performance gains across state-of-the-art trackers.

Contribution

The paper presents a novel, modular CP-based tracklet associator that can be integrated with existing trackers to improve their object association performance.

Findings

Improved HOTA and IDF1 scores by 3-4 points across tested trackers.

Effective handling of uncertain associations and tracklet overlaps.

Enhanced trajectory continuity with simple interpolation.

Abstract

Multiple object tracking (MOT) is a task in computer vision that aims to detect the position of various objects in videos and to associate them to a unique identity. We propose an approach based on Constraint Programming (CP) whose goal is to be grafted to any existing tracker in order to improve its object association results. We developed a modular algorithm divided into three independent phases. The first phase consists in recovering the tracklets provided by a base tracker and to cut them at the places where uncertain associations are spotted, for example, when tracklets overlap, which may cause identity switches. In the second phase, we associate the previously constructed tracklets using a Belief Propagation Constraint Programming algorithm, where we propose various constraints that assign scores to each of the tracklets based on multiple characteristics, such as their dynamics or the distance between them in time and space. Finally, the third phase is a rudimentary interpolation model to fill in the remaining holes in the trajectories we built. Experiments show that our model leads to improvements in the results for all three of the state-of-the-art trackers on which we tested it (3 to 4 points gained on HOTA and IDF1).