No Blind Spots: Full-Surround Multi-Object Tracking for Autonomous Vehicles using Cameras & LiDARs

TL;DR

This paper introduces a modular, sensor-fusion framework for full-surround multi-object tracking in autonomous vehicles, capable of tracking objects across multiple sensors and in real-world conditions, enhancing navigation and safety.

Contribution

It extends the MDP framework to integrate multiple sensor modalities and track objects directly in the real world, improving robustness and applicability for autonomous driving.

Findings

Effective multi-sensor fusion for object tracking demonstrated

Tracks objects through complex maneuvers around the vehicle

Modular approach allows analysis of sensor contributions

Abstract

Online multi-object tracking (MOT) is extremely important for high-level spatial reasoning and path planning for autonomous and highly-automated vehicles. In this paper, we present a modular framework for tracking multiple objects (vehicles), capable of accepting object proposals from different sensor modalities (vision and range) and a variable number of sensors, to produce continuous object tracks. This work is a generalization of the MDP framework for MOT, with some key extensions - First, we track objects across multiple cameras and across different sensor modalities. This is done by fusing object proposals across sensors accurately and efficiently. Second, the objects of interest (targets) are tracked directly in the real world. This is a departure from traditional techniques where objects are simply tracked in the image plane. Doing so allows the tracks to be readily used by an autonomous agent for navigation and related tasks. To verify the effectiveness of our approach, we test it on real world highway data collected from a heavily sensorized testbed capable of capturing full-surround information. We demonstrate that our framework is well-suited to track objects through entire maneuvers around the ego-vehicle, some of which take more than a few minutes to complete. We also leverage the modularity of our approach by comparing the effects of including/excluding different sensors, changing the total number of sensors, and the quality of object proposals on the final tracking result.