FusionSORT: Fusion Methods for Online Multi-object Visual Tracking

TL;DR

This paper explores four fusion methods combining motion, appearance, height-IoU, and confidence cues for data association in multi-object visual tracking, emphasizing the importance of fusion choice.

Contribution

It systematically evaluates different fusion strategies for data association, highlighting their impact on tracking performance in multi-object scenarios.

Findings

Fusion method choice significantly affects tracking accuracy.

Weighted sum and Kalman filter gating perform well across datasets.

Weak cues like height-IoU improve association robustness.

Abstract

In this work, we investigate four different fusion methods for associating detections to tracklets in multi-object visual tracking. In addition to considering strong cues such as motion and appearance information, we also consider weak cues such as height intersection-over-union (height-IoU) and tracklet confidence information in the data association using different fusion methods. These fusion methods include minimum, weighted sum based on IoU, Kalman filter (KF) gating, and hadamard product of costs due to the different cues. We conduct extensive evaluations on validation sets of MOT17, MOT20 and DanceTrack datasets, and find out that the choice of a fusion method is key for data association in multi-object visual tracking. We hope that this investigative work helps the computer vision research community to use the right fusion method for data association in multi-object visual tracking.