360$^\circ$ from a Single Camera: A Few-Shot Approach for LiDAR Segmentation

TL;DR

This paper introduces ImageTo360, a few-shot, label-efficient LiDAR segmentation method that leverages image-based predictions to pretrain a LiDAR network, achieving state-of-the-art results with minimal labeled data.

Contribution

The paper presents a novel image-to-360 approach that pretrains LiDAR segmentation models using image-based semantic predictions, reducing the need for extensive labeled LiDAR data.

Findings

Outperforms existing label-efficient methods in LiDAR segmentation.

Surpasses some fully-supervised segmentation networks in accuracy.

Modular design allows generalization across architectures.

Abstract

Deep learning applications on LiDAR data suffer from a strong domain gap when applied to different sensors or tasks. In order for these methods to obtain similar accuracy on different data in comparison to values reported on public benchmarks, a large scale annotated dataset is necessary. However, in practical applications labeled data is costly and time consuming to obtain. Such factors have triggered various research in label-efficient methods, but a large gap remains to their fully-supervised counterparts. Thus, we propose ImageTo360, an effective and streamlined few-shot approach to label-efficient LiDAR segmentation. Our method utilizes an image teacher network to generate semantic predictions for LiDAR data within a single camera view. The teacher is used to pretrain the LiDAR segmentation student network, prior to optional fine-tuning on 360$^\circ$ data. Our method is implemented in a modular manner on the point level and as such is generalizable to different architectures. We improve over the current state-of-the-art results for label-efficient methods and even surpass some traditional fully-supervised segmentation networks.