StreamMOTP: Streaming and Unified Framework for Joint 3D Multi-Object Tracking and Trajectory Prediction

TL;DR

StreamMOTP introduces a streaming, unified framework that jointly handles 3D multi-object tracking and trajectory prediction, leveraging long-term memory and coordinate encoding to improve accuracy and consistency in autonomous driving scenarios.

Contribution

The paper presents a novel streaming, unified framework with a memory bank and relative positional encoding for joint 3D tracking and prediction, addressing previous limitations.

Findings

Outperforms previous methods on nuScenes dataset

Improves trajectory prediction accuracy and consistency

Demonstrates potential for autonomous driving applications

Abstract

3D multi-object tracking and trajectory prediction are two crucial modules in autonomous driving systems. Generally, the two tasks are handled separately in traditional paradigms and a few methods have started to explore modeling these two tasks in a joint manner recently. However, these approaches suffer from the limitations of single-frame training and inconsistent coordinate representations between tracking and prediction tasks. In this paper, we propose a streaming and unified framework for joint 3D Multi-Object Tracking and trajectory Prediction (StreamMOTP) to address the above challenges. Firstly, we construct the model in a streaming manner and exploit a memory bank to preserve and leverage the long-term latent features for tracked objects more effectively. Secondly, a relative spatio-temporal positional encoding strategy is introduced to bridge the gap of coordinate representations between the two tasks and maintain the pose-invariance for trajectory prediction. Thirdly, we further improve the quality and consistency of predicted trajectories with a dual-stream predictor. We conduct extensive experiments on popular nuSences dataset and the experimental results demonstrate the effectiveness and superiority of StreamMOTP, which outperforms previous methods significantly on both tasks. Furthermore, we also prove that the proposed framework has great potential and advantages in actual applications of autonomous driving.