MapDistill: Boosting Efficient Camera-based HD Map Construction via Camera-LiDAR Fusion Model Distillation

TL;DR

MapDistill introduces a novel knowledge distillation framework that transfers knowledge from a camera-LiDAR fusion model to a lightweight camera-only model, significantly improving HD map construction efficiency and accuracy in autonomous driving.

Contribution

The paper proposes the first KD-based approach for HD map construction, employing a dual BEV transform and comprehensive distillation schemes to enable cost-effective camera-only models to outperform existing methods.

Findings

Surpasses existing methods by over 7.7 mAP on nuScenes

Achieves 4.5 times speedup in HD map construction

Effectively transfers knowledge from fusion to camera-only models

Abstract

Online high-definition (HD) map construction is an important and challenging task in autonomous driving. Recently, there has been a growing interest in cost-effective multi-view camera-based methods without relying on other sensors like LiDAR. However, these methods suffer from a lack of explicit depth information, necessitating the use of large models to achieve satisfactory performance. To address this, we employ the Knowledge Distillation (KD) idea for efficient HD map construction for the first time and introduce a novel KD-based approach called MapDistill to transfer knowledge from a high-performance camera-LiDAR fusion model to a lightweight camera-only model. Specifically, we adopt the teacher-student architecture, i.e., a camera-LiDAR fusion model as the teacher and a lightweight camera model as the student, and devise a dual BEV transform module to facilitate cross-modal knowledge distillation while maintaining cost-effective camera-only deployment. Additionally, we present a comprehensive distillation scheme encompassing cross-modal relation distillation, dual-level feature distillation, and map head distillation. This approach alleviates knowledge transfer challenges between modalities, enabling the student model to learn improved feature representations for HD map construction. Experimental results on the challenging nuScenes dataset demonstrate the effectiveness of MapDistill, surpassing existing competitors by over 7.7 mAP or 4.5X speedup.