An Empirical Analysis of Visual Features for Multiple Object Tracking in Urban Scenes

TL;DR

This study evaluates various appearance features and affinity measures for multiple object tracking in urban scenes, finding that ReID features outperform others in discriminating objects, with recommendations based on detector quality and occlusion levels.

Contribution

It provides a comprehensive empirical comparison of appearance features and affinity measures for MOT in urban environments, highlighting the superior performance of ReID features.

Findings

ReID features are the most effective for object discrimination in urban MOT.

Color histograms work well with high-recall detectors and minimal occlusion.

Deep features are more robust when detector recall is low.

Abstract

This paper addresses the problem of selecting appearance features for multiple object tracking (MOT) in urban scenes. Over the years, a large number of features has been used for MOT. However, it is not clear whether some of them are better than others. Commonly used features are color histograms, histograms of oriented gradients, deep features from convolutional neural networks and re-identification (ReID) features. In this study, we assess how good these features are at discriminating objects enclosed by a bounding box in urban scene tracking scenarios. Several affinity measures, namely the $\mathrm{L}_1$, $\mathrm{L}_2$ and the Bhattacharyya distances, Rank-1 counts and the cosine similarity, are also assessed for their impact on the discriminative power of the features. Results on several datasets show that features from ReID networks are the best for discriminating instances from one another regardless of the quality of the detector. If a ReID model is not available, color histograms may be selected if the detector has a good recall and there are few occlusions; otherwise, deep features are more robust to detectors with lower recall. The project page is http://www.mehdimiah.com/visual_features.