MotionTrack: End-to-End Transformer-based Multi-Object Tracing with LiDAR-Camera Fusion

TL;DR

MotionTrack introduces an end-to-end transformer-based multi-object tracking algorithm utilizing LiDAR and camera data, demonstrating improved performance in autonomous driving scenarios by combining detection and tracking in a unified framework.

Contribution

This paper presents a novel transformer-based MOT method with multi-modality inputs, establishing a new baseline for autonomous vehicle perception tasks.

Findings

Achieves an AMOTA score of 0.55 on nuScenes dataset.

Outperforms classical models like AB3DMOT and CenterTrack.

Utilizes a modified attention mechanism for data association.

Abstract

Multiple Object Tracking (MOT) is crucial to autonomous vehicle perception. End-to-end transformer-based algorithms, which detect and track objects simultaneously, show great potential for the MOT task. However, most existing methods focus on image-based tracking with a single object category. In this paper, we propose an end-to-end transformer-based MOT algorithm (MotionTrack) with multi-modality sensor inputs to track objects with multiple classes. Our objective is to establish a transformer baseline for the MOT in an autonomous driving environment. The proposed algorithm consists of a transformer-based data association (DA) module and a transformer-based query enhancement module to achieve MOT and Multiple Object Detection (MOD) simultaneously. The MotionTrack and its variations achieve better results (AMOTA score at 0.55) on the nuScenes dataset compared with other classical baseline models, such as the AB3DMOT, the CenterTrack, and the probabilistic 3D Kalman filter. In addition, we prove that a modified attention mechanism can be utilized for DA to accomplish the MOT, and aggregate history features to enhance the MOD performance.