Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking

TL;DR

OC-SORT enhances multi-object tracking by integrating object observations to correct Kalman filter errors during occlusion, significantly improving robustness and accuracy in non-linear motion scenarios while maintaining real-time performance.

Contribution

This work introduces Observation-Centric SORT (OC-SORT), a novel approach that uses object observations to fix error accumulation in Kalman filters during occlusion, improving robustness in MOT.

Findings

Achieves state-of-the-art results on multiple datasets.

Runs at over 700 FPS on a single CPU.

Excels in tracking highly non-linear object motions.

Abstract

Kalman filter (KF) based methods for multi-object tracking (MOT) make an assumption that objects move linearly. While this assumption is acceptable for very short periods of occlusion, linear estimates of motion for prolonged time can be highly inaccurate. Moreover, when there is no measurement available to update Kalman filter parameters, the standard convention is to trust the priori state estimations for posteriori update. This leads to the accumulation of errors during a period of occlusion. The error causes significant motion direction variance in practice. In this work, we show that a basic Kalman filter can still obtain state-of-the-art tracking performance if proper care is taken to fix the noise accumulated during occlusion. Instead of relying only on the linear state estimate (i.e., estimation-centric approach), we use object observations (i.e., the measurements by object detector) to compute a virtual trajectory over the occlusion period to fix the error accumulation of filter parameters during the occlusion period. This allows more time steps to correct errors accumulated during occlusion. We name our method Observation-Centric SORT (OC-SORT). It remains Simple, Online, and Real-Time but improves robustness during occlusion and non-linear motion. Given off-the-shelf detections as input, OC-SORT runs at 700+ FPS on a single CPU. It achieves state-of-the-art on multiple datasets, including MOT17, MOT20, KITTI, head tracking, and especially DanceTrack where the object motion is highly non-linear. The code and models are available at \url{https://github.com/noahcao/OC_SORT}.