Multi-modal curb detection and filtering

TL;DR

This paper presents a multi-modal curb detection method combining camera semantics and dense lidar point clouds, enabling robust detection of multiple curbs in complex urban environments for autonomous vehicles.

Contribution

It introduces a novel fusion of lidar and camera data, along with clustering and filtering techniques, to improve curb detection accuracy and robustness in diverse urban scenarios.

Findings

Effective detection of curbs in complex urban scenes

Robust multi-curb detection across multiple lanes

Comparison shows Delaunay filtering outperforms RANSAC

Abstract

Reliable knowledge of road boundaries is critical for autonomous vehicle navigation. We propose a robust curb detection and filtering technique based on the fusion of camera semantics and dense lidar point clouds. The lidar point clouds are collected by fusing multiple lidars for robust feature detection. The camera semantics are based on a modified EfficientNet architecture which is trained with labeled data collected from onboard fisheye cameras. The point clouds are associated with the closest curb segment with $L_2$-norm analysis after projecting into the image space with the fisheye model projection. Next, the selected points are clustered using unsupervised density-based spatial clustering to detect different curb regions. As new curb points are detected in consecutive frames they are associated with the existing curb clusters using temporal reachability constraints. If no reachability constraints are found a new curb cluster is formed from these new points. This ensures we can detect multiple curbs present in road segments consisting of multiple lanes if they are in the sensors' field of view. Finally, Delaunay filtering is applied for outlier removal and its performance is compared to traditional RANSAC-based filtering. An objective evaluation of the proposed solution is done using a high-definition map containing ground truth curb points obtained from a commercial map supplier. The proposed system has proven capable of detecting curbs of any orientation in complex urban road scenarios comprising straight roads, curved roads, and intersections with traffic isles.