DeepFusionMOT: A 3D Multi-Object Tracking Framework Based on Camera-LiDAR Fusion with Deep Association

TL;DR

DeepFusionMOT introduces a camera-LiDAR fusion approach for 3D multi-object tracking that balances high accuracy with computational efficiency by leveraging a deep association mechanism for effective data integration.

Contribution

It presents a novel fusion-based MOT framework with a deep association mechanism that improves tracking accuracy and speed over existing methods.

Findings

Outperforms state-of-the-art methods in accuracy and speed

Effective in tracking objects with limited LiDAR data

Achieves smooth fusion of 2D and 3D trajectories

Abstract

In the recent literature, on the one hand, many 3D multi-object tracking (MOT) works have focused on tracking accuracy and neglected computation speed, commonly by designing rather complex cost functions and feature extractors. On the other hand, some methods have focused too much on computation speed at the expense of tracking accuracy. In view of these issues, this paper proposes a robust and fast camera-LiDAR fusion-based MOT method that achieves a good trade-off between accuracy and speed. Relying on the characteristics of camera and LiDAR sensors, an effective deep association mechanism is designed and embedded in the proposed MOT method. This association mechanism realizes tracking of an object in a 2D domain when the object is far away and only detected by the camera, and updating of the 2D trajectory with 3D information obtained when the object appears in the LiDAR field of view to achieve a smooth fusion of 2D and 3D trajectories. Extensive experiments based on the typical datasets indicate that our proposed method presents obvious advantages over the state-of-the-art MOT methods in terms of both tracking accuracy and processing speed. Our code is made publicly available for the benefit of the community.