A Joint 3D-2D based Method for Free Space Detection on Roads

TL;DR

This paper introduces a novel method combining 2D CNN-based road segmentation with 3D SLAM data to improve free space detection for autonomous driving, addressing limitations of purely 2D or 3D approaches.

Contribution

It proposes a joint 3D-2D approach using CNNs and CRF with SLAM data for more accurate road and free space detection in complex environments.

Findings

Outperforms state-of-the-art methods on KITTI and Camvid datasets

Effectively handles uneven textures and shadows in road scenes

Improves free space detection accuracy for autonomous navigation

Abstract

In this paper, we address the problem of road segmentation and free space detection in the context of autonomous driving. Traditional methods either use 3-dimensional (3D) cues such as point clouds obtained from LIDAR, RADAR or stereo cameras or 2-dimensional (2D) cues such as lane markings, road boundaries and object detection. Typical 3D point clouds do not have enough resolution to detect fine differences in heights such as between road and pavement. Image based 2D cues fail when encountering uneven road textures such as due to shadows, potholes, lane markings or road restoration. We propose a novel free road space detection technique combining both 2D and 3D cues. In particular, we use CNN based road segmentation from 2D images and plane/box fitting on sparse depth data obtained from SLAM as priors to formulate an energy minimization using conditional random field (CRF), for road pixels classification. While the CNN learns the road texture and is unaffected by depth boundaries, the 3D information helps in overcoming texture based classification failures. Finally, we use the obtained road segmentation with the 3D depth data from monocular SLAM to detect the free space for the navigation purposes. Our experiments on KITTI odometry dataset, Camvid dataset, as well as videos captured by us, validate the superiority of the proposed approach over the state of the art.