High-level camera-LiDAR fusion for 3D object detection with machine learning

TL;DR

This paper presents a machine learning framework combining camera and LiDAR data for 3D vehicle detection, utilizing frustum proposals and classical ML algorithms to achieve high accuracy in autonomous driving scenarios.

Contribution

It introduces a novel high-level fusion approach using ML on camera-LiDAR data with frustum proposals for improved 3D object detection.

Findings

Achieved 87.1% overall accuracy in vehicle detection

Effective segmentation of LiDAR point clouds using 2D detector proposals

Demonstrated efficient and accurate inference pipeline

Abstract

This paper tackles the 3D object detection problem, which is of vital importance for applications such as autonomous driving. Our framework uses a Machine Learning (ML) pipeline on a combination of monocular camera and LiDAR data to detect vehicles in the surrounding 3D space of a moving platform. It uses frustum region proposals generated by State-Of-The-Art (SOTA) 2D object detectors to segment LiDAR point clouds into point clusters which represent potentially individual objects. We evaluate the performance of classical ML algorithms as part of an holistic pipeline for estimating the parameters of 3D bounding boxes which surround the vehicles around the moving platform. Our results demonstrate an efficient and accurate inference on a validation set, achieving an overall accuracy of 87.1%.