Pseudo-LiDAR Based Road Detection

TL;DR

This paper introduces a novel RGB-only road detection method that uses pseudo-LiDAR and NAS-optimized feature fusion, achieving state-of-the-art results without relying on LiDAR sensors during inference.

Contribution

The paper presents a new RGB-based road detection approach employing pseudo-LiDAR, NAS for fusion network optimization, and a modality distillation strategy to eliminate extra computational costs.

Findings

Achieves state-of-the-art performance on KITTI and R2D benchmarks.

Effectively fuses RGB and depth information for improved detection.

Reduces inference costs by removing dependency on depth estimation networks.

Abstract

Road detection is a critically important task for self-driving cars. By employing LiDAR data, recent works have significantly improved the accuracy of road detection. Relying on LiDAR sensors limits the wide application of those methods when only cameras are available. In this paper, we propose a novel road detection approach with RGB being the only input during inference. Specifically, we exploit pseudo-LiDAR using depth estimation, and propose a feature fusion network where RGB and learned depth information are fused for improved road detection. To further optimize the network structure and improve the efficiency of the network. we search for the network structure of the feature fusion module using NAS techniques. Finally, be aware of that generating pseudo-LiDAR from RGB via depth estimation introduces extra computational costs and relies on depth estimation networks, we design a modality distillation strategy and leverage it to further free our network from these extra computational cost and dependencies during inference. The proposed method achieves state-of-the-art performance on two challenging benchmarks, KITTI and R2D.