FeatureSORT: Essential Features for Effective Tracking

TL;DR

FeatureSORT enhances multi-object tracking by integrating multiple appearance attributes and advanced post-processing, significantly improving identity preservation and accuracy on standard benchmarks.

Contribution

The paper introduces FeatureSORT, a novel online tracker that extends DeepSORT with multi-attribute detection and improved association strategies.

Findings

Achieves state-of-the-art MOTA scores on MOT benchmarks.

Effectively maintains identities through occlusions.

Reduces identity switches compared to previous methods.

Abstract

We introduce FeatureSORT, a simple yet effective online multiple object tracker that reinforces the DeepSORT baseline with a redesigned detector and additional feature cues. In contrast to conventional detectors that only provide bounding boxes, our modified YOLOX architecture is extended to output multiple appearance attributes, including clothing color, clothing style, and motion direction, alongside the bounding boxes. These feature cues, together with a ReID network, form complementary embeddings that substantially improve association accuracy. Furthermore, we incorporate stronger post-processing strategies, such as global linking and Gaussian Smoothing Process interpolation, to handle missing associations and detections. During online tracking, we define a measurement-to-track distance function that jointly considers IoU, direction, color, style, and ReID similarity. This design enables FeatureSORT to maintain consistent identities through longer occlusions while reducing identity switches. Extensive experiments on standard MOT benchmarks demonstrate that FeatureSORT achieves state-of-the-art online performance, with MOTA scores of 79.7 on MOT16, 80.6 on MOT17, 77.9 on MOT20, and 92.2 on DanceTrack, underscoring the effectiveness of feature-enriched detection and modular post processing in advancing multi-object tracking.