JDT3D: Addressing the Gaps in LiDAR-Based Tracking-by-Attention

TL;DR

JDT3D introduces novel methods to improve LiDAR-based tracking-by-attention, significantly narrowing the performance gap with tracking-by-detection in autonomous driving scenarios.

Contribution

The paper proposes track sampling augmentation and confidence-based query propagation to enhance TBA methods, validated by superior results on the nuScenes dataset.

Findings

JDT3D outperforms existing LiDAR-based TBA methods by over 6% on AMOTA.

The proposed methods improve long occlusion tracking in LiDAR-based systems.

Analysis reveals challenges in current TBA models that hinder performance.

Abstract

Tracking-by-detection (TBD) methods achieve state-of-the-art performance on 3D tracking benchmarks for autonomous driving. On the other hand, tracking-by-attention (TBA) methods have the potential to outperform TBD methods, particularly for long occlusions and challenging detection settings. This work investigates why TBA methods continue to lag in performance behind TBD methods using a LiDAR-based joint detector and tracker called JDT3D. Based on this analysis, we propose two generalizable methods to bridge the gap between TBD and TBA methods: track sampling augmentation and confidence-based query propagation. JDT3D is trained and evaluated on the nuScenes dataset, achieving 0.574 on the AMOTA metric on the nuScenes test set, outperforming all existing LiDAR-based TBA approaches by over 6%. Based on our results, we further discuss some potential challenges with the existing TBA model formulation to explain the continued gap in performance with TBD methods. The implementation of JDT3D can be found at the following link: https://github.com/TRAILab/JDT3D.