Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentation

TL;DR

This paper introduces Point-to-Voxel Knowledge Distillation (PVD), a novel method for LiDAR semantic segmentation that effectively transfers knowledge from large models to smaller ones by leveraging point and voxel level information, structural affinity, and a difficulty-aware sampling strategy.

Contribution

The paper proposes a new distillation approach that combines pointwise and voxelwise supervision with affinity-based structural information and a difficulty-aware sampling strategy for improved LiDAR segmentation.

Findings

PVD outperforms previous distillation methods on nuScenes and SemanticKITTI.

Achieves roughly 75% MACs reduction and 2x speedup on Cylinder3D.

Ranks 1st on SemanticKITTI leaderboard among published algorithms.

Abstract

This article addresses the problem of distilling knowledge from a large teacher model to a slim student network for LiDAR semantic segmentation. Directly employing previous distillation approaches yields inferior results due to the intrinsic challenges of point cloud, i.e., sparsity, randomness and varying density. To tackle the aforementioned problems, we propose the Point-to-Voxel Knowledge Distillation (PVD), which transfers the hidden knowledge from both point level and voxel level. Specifically, we first leverage both the pointwise and voxelwise output distillation to complement the sparse supervision signals. Then, to better exploit the structural information, we divide the whole point cloud into several supervoxels and design a difficulty-aware sampling strategy to more frequently sample supervoxels containing less-frequent classes and faraway objects. On these supervoxels, we propose inter-point and inter-voxel affinity distillation, where the similarity information between points and voxels can help the student model better capture the structural information of the surrounding environment. We conduct extensive experiments on two popular LiDAR segmentation benchmarks, i.e., nuScenes and SemanticKITTI. On both benchmarks, our PVD consistently outperforms previous distillation approaches by a large margin on three representative backbones, i.e., Cylinder3D, SPVNAS and MinkowskiNet. Notably, on the challenging nuScenes and SemanticKITTI datasets, our method can achieve roughly 75% MACs reduction and 2x speedup on the competitive Cylinder3D model and rank 1st on the SemanticKITTI leaderboard among all published algorithms. Our code is available at https://github.com/cardwing/Codes-for-PVKD.