KD360-VoxelBEV: LiDAR and 360-degree Camera Cross Modality Knowledge Distillation for Bird's-Eye-View Segmentation

TL;DR

This paper introduces a cross-modality knowledge distillation framework that enhances camera-only Bird's-Eye-View segmentation by leveraging LiDAR data during training, resulting in improved accuracy and efficiency for autonomous driving applications.

Contribution

It presents the first cross-modality distillation method for single-panoramic-camera BEV segmentation, combining LiDAR and camera data to improve performance and generalize across sensor setups.

Findings

Teacher model achieves 25.6% IoU improvement over existing methods.

Student network gains 8.5% IoU improvement and runs at 31.2 FPS.

Framework generalizes well to different camera configurations like KITTI-360.

Abstract

We present the first cross-modality distillation framework specifically tailored for single-panoramic-camera Bird's-Eye-View (BEV) segmentation. Our approach leverages a novel LiDAR image representation fused from range, intensity and ambient channels, together with a voxel-aligned view transformer that preserves spatial fidelity while enabling efficient BEV processing. During training, a high-capacity LiDAR and camera fusion Teacher network extracts both rich spatial and semantic features for cross-modality knowledge distillation into a lightweight Student network that relies solely on a single 360-degree panoramic camera image. Extensive experiments on the Dur360BEV dataset demonstrate that our teacher model significantly outperforms existing camera-based BEV segmentation methods, achieving a 25.6\% IoU improvement. Meanwhile, the distilled Student network attains competitive performance with an 8.5\% IoU gain and state-of-the-art inference speed of 31.2 FPS. Moreover, evaluations on KITTI-360 (two fisheye cameras) confirm that our distillation framework generalises to diverse camera setups, underscoring its feasibility and robustness. This approach reduces sensor complexity and deployment costs while providing a practical solution for efficient, low-cost BEV segmentation in real-world autonomous driving.