RockTrack: A 3D Robust Multi-Camera-Ken Multi-Object Tracking Framework

TL;DR

RockTrack is a versatile 3D multi-camera multi-object tracking framework that effectively fuses geometric and visual cues, achieving state-of-the-art results with high efficiency.

Contribution

It introduces a detector-agnostic tracking method with confidence-guided preprocessing, a novel appearance similarity metric, and a staged association process for improved multi-camera 3D MOT.

Findings

Achieves 59.1% AMOTA on nuScenes leaderboard.

Demonstrates high computational efficiency.

Compatible with various off-the-shelf detectors.

Abstract

3D Multi-Object Tracking (MOT) obtains significant performance improvements with the rapid advancements in 3D object detection, particularly in cost-effective multi-camera setups. However, the prevalent end-to-end training approach for multi-camera trackers results in detector-specific models, limiting their versatility. Moreover, current generic trackers overlook the unique features of multi-camera detectors, i.e., the unreliability of motion observations and the feasibility of visual information. To address these challenges, we propose RockTrack, a 3D MOT method for multi-camera detectors. Following the Tracking-By-Detection framework, RockTrack is compatible with various off-the-shelf detectors. RockTrack incorporates a confidence-guided preprocessing module to extract reliable motion and image observations from distinct representation spaces from a single detector. These observations are then fused in an association module that leverages geometric and appearance cues to minimize mismatches. The resulting matches are propagated through a staged estimation process, forming the basis for heuristic noise modeling. Additionally, we introduce a novel appearance similarity metric for explicitly characterizing object affinities in multi-camera settings. RockTrack achieves state-of-the-art performance on the nuScenes vision-only tracking leaderboard with 59.1% AMOTA while demonstrating impressive computational efficiency.