DRUM: Diffusion-based Raydrop-aware Unpaired Mapping for Sim2Real LiDAR Segmentation

TL;DR

DRUM is a diffusion-based framework that enhances synthetic-to-real LiDAR segmentation by translating synthetic data to match real-world measurement characteristics, improving model performance.

Contribution

It introduces a novel diffusion model-based translation method that reproduces key LiDAR measurement features for better domain adaptation.

Findings

DRUM improves Sim2Real LiDAR segmentation accuracy.

The method effectively reproduces reflectance and raydrop noise characteristics.

Experimental results show consistent performance gains across datasets.

Abstract

LiDAR-based semantic segmentation is a key component for autonomous mobile robots, yet large-scale annotation of LiDAR point clouds is prohibitively expensive and time-consuming. Although simulators can provide labeled synthetic data, models trained on synthetic data often underperform on real-world data due to a data-level domain gap. To address this issue, we propose DRUM, a novel Sim2Real translation framework. We leverage a diffusion model pre-trained on unlabeled real-world data as a generative prior and translate synthetic data by reproducing two key measurement characteristics: reflectance intensity and raydrop noise. To improve sample fidelity, we introduce a raydrop-aware masked guidance mechanism that selectively enforces consistency with the input synthetic data while preserving realistic raydrop noise induced by the diffusion prior. Experimental results demonstrate that DRUM consistently improves Sim2Real performance across multiple representations of LiDAR data. The project page is available at https://miya-tomoya.github.io/drum.