A Pedestrian Detection and Tracking Framework for Autonomous Cars: Efficient Fusion of Camera and LiDAR Data

TL;DR

This paper introduces an integrated pedestrian detection and tracking framework for autonomous vehicles that fuses camera and LiDAR data, utilizing deep learning and Kalman filtering to improve accuracy in complex driving scenarios.

Contribution

The paper proposes a novel fusion approach combining depth images from LiDAR and RGB data with deep neural networks and Kalman filtering for enhanced pedestrian detection and tracking.

Findings

Significant performance improvement over image-only methods

Effective fusion of camera and LiDAR data for detection

Robust multi-pedestrian tracking in real driving scenarios

Abstract

This paper presents a novel method for pedestrian detection and tracking by fusing camera and LiDAR sensor data. To deal with the challenges associated with the autonomous driving scenarios, an integrated tracking and detection framework is proposed. The detection phase is performed by converting LiDAR streams to computationally tractable depth images, and then, a deep neural network is developed to identify pedestrian candidates both in RGB and depth images. To provide accurate information, the detection phase is further enhanced by fusing multi-modal sensor information using the Kalman filter. The tracking phase is a combination of the Kalman filter prediction and an optical flow algorithm to track multiple pedestrians in a scene. We evaluate our framework on a real public driving dataset. Experimental results demonstrate that the proposed method achieves significant performance improvement over a baseline method that solely uses image-based pedestrian detection.