LiDAR MOT-DETR: A LiDAR-based Two-Stage Transformer for 3D Multiple Object Tracking

TL;DR

LiDAR MOT-DETR introduces a two-stage transformer approach for 3D multi-object tracking that refines detections and maintains object identities over time, outperforming existing models on nuScenes and KITTI datasets.

Contribution

The paper proposes a novel two-stage transformer framework for LiDAR-based 3D tracking, combining detection refinement and temporal association using attention mechanisms.

Findings

Online mode achieves higher accuracy than baseline and SOTA models.

Offline mode improves tracking precision by 3 percentage points.

Model demonstrates strong performance on nuScenes and KITTI datasets.

Abstract

Multi-object tracking from LiDAR point clouds presents unique challenges due to the sparse and irregular nature of the data, compounded by the need for temporal coherence across frames. Traditional tracking systems often rely on hand-crafted features and motion models, which can struggle to maintain consistent object identities in crowded or fast-moving scenes. We present a lidar-based two-staged DETR inspired transformer; a smoother and tracker. The smoother stage refines lidar object detections, from any off-the-shelf detector, across a moving temporal window. The tracker stage uses a DETR-based attention block to maintain tracks across time by associating tracked objects with the refined detections using the point cloud as context. The model is trained on the datasets nuScenes and KITTI in both online and offline (forward peeking) modes demonstrating strong performance across metrics such as ID-switch and multiple object tracking accuracy (MOTA). The numerical results indicate that the online mode outperforms the lidar-only baseline and SOTA models on the nuScenes dataset, with an aMOTA of 0.724 and an aMOTP of 0.475, while the offline mode provides an additional 3 pp aMOTP.