LiDAR2Map: In Defense of LiDAR-Based Semantic Map Construction Using Online Camera Distillation

TL;DR

LiDAR2Map introduces a novel LiDAR-based semantic map construction method enhanced by a BEV feature pyramid decoder and online camera-to-LiDAR distillation, significantly improving accuracy and semantic understanding in autonomous driving.

Contribution

The paper proposes a BEV feature pyramid decoder and an online camera-to-LiDAR distillation scheme to enhance LiDAR-based semantic map construction, addressing the lack of semantic cues in LiDAR data.

Findings

Achieves over 27.9% mIoU improvement on nuScenes dataset.

Outperforms previous LiDAR-based methods and surpasses camera-based approaches.

Demonstrates effective semantic learning transfer from camera to LiDAR.

Abstract

Semantic map construction under bird's-eye view (BEV) plays an essential role in autonomous driving. In contrast to camera image, LiDAR provides the accurate 3D observations to project the captured 3D features onto BEV space inherently. However, the vanilla LiDAR-based BEV feature often contains many indefinite noises, where the spatial features have little texture and semantic cues. In this paper, we propose an effective LiDAR-based method to build semantic map. Specifically, we introduce a BEV feature pyramid decoder that learns the robust multi-scale BEV features for semantic map construction, which greatly boosts the accuracy of the LiDAR-based method. To mitigate the defects caused by lacking semantic cues in LiDAR data, we present an online Camera-to-LiDAR distillation scheme to facilitate the semantic learning from image to point cloud. Our distillation scheme consists of feature-level and logit-level distillation to absorb the semantic information from camera in BEV. The experimental results on challenging nuScenes dataset demonstrate the efficacy of our proposed LiDAR2Map on semantic map construction, which significantly outperforms the previous LiDAR-based methods over 27.9% mIoU and even performs better than the state-of-the-art camera-based approaches. Source code is available at: https://github.com/songw-zju/LiDAR2Map.