From One to the Power of Many: Invariance to Multi-LiDAR Perception from Single-Sensor Datasets

TL;DR

This paper introduces a new metric and data augmentation techniques to improve the generalization of LiDAR segmentation models trained on single sensors when applied to multi-sensor setups in autonomous vehicles.

Contribution

It proposes a feature-level invariance metric and two data augmentations to enhance cross-sensor generalization without needing labeled multi-sensor data.

Findings

Proposed invariance metric correlates with cross-sensor performance.

Data augmentations improve model transferability to multi-sensor LiDAR setups.

Experimental results show enhanced invariance and generalization on simulated and real data.

Abstract

Recently, LiDAR segmentation methods for autonomous vehicles, powered by deep neural networks, have experienced steep growth in performance on classic benchmarks, such as nuScenes and SemanticKITTI. However, there are still large gaps in performance when deploying models trained on such single-sensor setups to modern vehicles with multiple high-resolution LiDAR sensors. In this work, we introduce a new metric for feature-level invariance which can serve as a proxy to measure cross-domain generalization without requiring labeled data. Additionally, we propose two application-specific data augmentations, which facilitate better transfer to multi-sensor LiDAR setups, when trained on single-sensor datasets. We provide experimental evidence on both simulated and real data, that our proposed augmentations improve invariance across LiDAR setups, leading to improved generalization.