3D Multi-Object Tracking Using Graph Neural Networks with Cross-Edge Modality Attention

TL;DR

This paper introduces Batch3DMOT, a novel 3D multi-object tracking method using graph neural networks with cross-edge attention to effectively integrate multi-modal sensor data, achieving state-of-the-art results on nuScenes and KITTI datasets.

Contribution

The paper proposes a multi-modal graph neural network with cross-edge attention for 3D MOT, addressing modality intermittence and enabling information exchange across disconnected graph components.

Findings

Achieves 3.3% improvement in AMOTA on nuScenes

Sets new state-of-the-art for 3D tracking

Enhances false positive filtering

Abstract

Online 3D multi-object tracking (MOT) has witnessed significant research interest in recent years, largely driven by demand from the autonomous systems community. However, 3D offline MOT is relatively less explored. Labeling 3D trajectory scene data at a large scale while not relying on high-cost human experts is still an open research question. In this work, we propose Batch3DMOT which follows the tracking-by-detection paradigm and represents real-world scenes as directed, acyclic, and category-disjoint tracking graphs that are attributed using various modalities such as camera, LiDAR, and radar. We present a multi-modal graph neural network that uses a cross-edge attention mechanism mitigating modality intermittence, which translates into sparsity in the graph domain. Additionally, we present attention-weighted convolutions over frame-wise k-NN neighborhoods as suitable means to allow information exchange across disconnected graph components. We evaluate our approach using various sensor modalities and model configurations on the challenging nuScenes and KITTI datasets. Extensive experiments demonstrate that our proposed approach yields an overall improvement of 3.3% in the AMOTA score on nuScenes thereby setting the new state-of-the-art for 3D tracking and further enhancing false positive filtering.