PD-SORT: Occlusion-Robust Multi-Object Tracking Using Pseudo-Depth Cues

TL;DR

PD-SORT introduces pseudo-depth cues and novel association strategies to improve multi-object tracking robustness in occluded and dynamic scenes, achieving state-of-the-art results especially in complex motion scenarios.

Contribution

The paper proposes PD-SORT, a multi-object tracking method that incorporates pseudo-depth cues, depth volume IoU, and camera motion compensation for enhanced occlusion handling.

Findings

Significantly improves tracking accuracy in occluded scenes

Achieves leading performance on DanceTrack, MOT17, and MOT20 datasets

Especially effective in complex motion and occlusion scenarios

Abstract

Multi-object tracking (MOT) is a rising topic in video processing technologies and has important application value in consumer electronics. Currently, tracking-by-detection (TBD) is the dominant paradigm for MOT, which performs target detection and association frame by frame. However, the association performance of TBD methods degrades in complex scenes with heavy occlusions, which hinders the application of such methods in real-world scenarios.To this end, we incorporate pseudo-depth cues to enhance the association performance and propose Pseudo-Depth SORT (PD-SORT). First, we extend the Kalman filter state vector with pseudo-depth states. Second, we introduce a novel depth volume IoU (DVIoU) by combining the conventional 2D IoU with pseudo-depth. Furthermore, we develop a quantized pseudo-depth measurement (QPDM) strategy for more robust data association. Besides, we also integrate camera motion compensation (CMC) to handle dynamic camera situations. With the above designs, PD-SORT significantly alleviates the occlusion-induced ambiguous associations and achieves leading performances on DanceTrack, MOT17, and MOT20. Note that the improvement is especially obvious on DanceTrack, where objects show complex motions, similar appearances, and frequent occlusions. The code is available at https://github.com/Wangyc2000/PD_SORT.