Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Driving

TL;DR

This paper introduces a probabilistic, multi-modal multi-object tracking system for autonomous driving that fuses image and LiDAR data, learns data association metrics, and improves track management, outperforming existing methods on standard datasets.

Contribution

The paper presents a novel trainable, probabilistic tracking framework that integrates multi-modal data and learns key components for improved accuracy in autonomous vehicle tracking.

Findings

Outperforms state-of-the-art methods on NuScenes and KITTI datasets.

Effectively fuses 2D and 3D data for robust tracking.

Learns data association metrics and track initialization strategies.

Abstract

Multi-object tracking is an important ability for an autonomous vehicle to safely navigate a traffic scene. Current state-of-the-art follows the tracking-by-detection paradigm where existing tracks are associated with detected objects through some distance metric. The key challenges to increase tracking accuracy lie in data association and track life cycle management. We propose a probabilistic, multi-modal, multi-object tracking system consisting of different trainable modules to provide robust and data-driven tracking results. First, we learn how to fuse features from 2D images and 3D LiDAR point clouds to capture the appearance and geometric information of an object. Second, we propose to learn a metric that combines the Mahalanobis and feature distances when comparing a track and a new detection in data association. And third, we propose to learn when to initialize a track from an unmatched object detection. Through extensive quantitative and qualitative results, we show that when using the same object detectors our method outperforms state-of-the-art approaches on the NuScenes and KITTI datasets.