MOTCOM: The Multi-Object Tracking Dataset Complexity Metric

TL;DR

The paper introduces MOTCOM, a new metric for quantifying the complexity of multi-object tracking sequences, addressing the lack of comprehensive measures and enabling better evaluation and comparison of tracker performance.

Contribution

MOTCOM is a novel complexity metric combining occlusion, erratic motion, and visual similarity, improving the understanding of dataset difficulty in MOT tasks.

Findings

MOTCOM outperforms traditional density and track count metrics in describing sequence complexity.

Evaluation on MOT17, MOT20, and MOTSynth datasets demonstrates MOTCOM's effectiveness.

MOTCOM facilitates nuanced analysis of tracker performance across different datasets.

Abstract

There exists no comprehensive metric for describing the complexity of Multi-Object Tracking (MOT) sequences. This lack of metrics decreases explainability, complicates comparison of datasets, and reduces the conversation on tracker performance to a matter of leader board position. As a remedy, we present the novel MOT dataset complexity metric (MOTCOM), which is a combination of three sub-metrics inspired by key problems in MOT: occlusion, erratic motion, and visual similarity. The insights of MOTCOM can open nuanced discussions on tracker performance and may lead to a wider acknowledgement of novel contributions developed for either less known datasets or those aimed at solving sub-problems. We evaluate MOTCOM on the comprehensive MOT17, MOT20, and MOTSynth datasets and show that MOTCOM is far better at describing the complexity of MOT sequences compared to the conventional density and number of tracks. Project page at https://vap.aau.dk/motcom