GRASPTrack: Geometry-Reasoned Association via Segmentation and Projection for Multi-Object Tracking

TL;DR

GRASPTrack introduces a depth-aware multi-object tracking framework that leverages monocular depth estimation and 3D geometric reasoning to improve robustness in occluded and complex scenes.

Contribution

It integrates monocular depth estimation with segmentation into a tracking pipeline, enabling explicit 3D reasoning and robust association through novel voxel-based IoU and adaptive noise compensation.

Findings

Achieves competitive performance on MOT17, MOT20, and DanceTrack benchmarks.

Significantly improves tracking robustness in occlusion-heavy scenes.

Enhances motion association with 3D motion cues.

Abstract

Multi-object tracking (MOT) in monocular videos is fundamentally challenged by occlusions and depth ambiguity, issues that conventional tracking-by-detection (TBD) methods struggle to resolve owing to a lack of geometric awareness. To address these limitations, we introduce GRASPTrack, a novel depth-aware MOT framework that integrates monocular depth estimation and instance segmentation into a standard TBD pipeline to generate high-fidelity 3D point clouds from 2D detections, thereby enabling explicit 3D geometric reasoning. These 3D point clouds are then voxelized to enable a precise and robust Voxel-Based 3D Intersection-over-Union (IoU) for spatial association. To further enhance tracking robustness, our approach incorporates Depth-aware Adaptive Noise Compensation, which dynamically adjusts the Kalman filter process noise based on occlusion severity for more reliable state estimation. Additionally, we propose a Depth-enhanced Observation-Centric Momentum, which extends the motion direction consistency from the image plane into 3D space to improve motion-based association cues, particularly for objects with complex trajectories. Extensive experiments on the MOT17, MOT20, and DanceTrack benchmarks demonstrate that our method achieves competitive performance, significantly improving tracking robustness in complex scenes with frequent occlusions and intricate motion patterns.