Generative Range Imaging for Learning Scene Priors of 3D LiDAR Data

TL;DR

This paper introduces a generative model for LiDAR range images that improves domain transfer, enhances data fidelity, and supports applications like upsampling, restoration, and realistic ray-drop simulation for autonomous robot perception.

Contribution

It presents a novel implicit image representation-based GAN with differentiable ray-drop modeling for LiDAR data, addressing domain gap and noise issues in perception tasks.

Findings

The model produces high-fidelity, diverse LiDAR range images.

It effectively upscales and restores LiDAR data.

It outperforms existing methods in Sim2Real semantic segmentation.

Abstract

3D LiDAR sensors are indispensable for the robust vision of autonomous mobile robots. However, deploying LiDAR-based perception algorithms often fails due to a domain gap from the training environment, such as inconsistent angular resolution and missing properties. Existing studies have tackled the issue by learning inter-domain mapping, while the transferability is constrained by the training configuration and the training is susceptible to peculiar lossy noises called ray-drop. To address the issue, this paper proposes a generative model of LiDAR range images applicable to the data-level domain transfer. Motivated by the fact that LiDAR measurement is based on point-by-point range imaging, we train an implicit image representation-based generative adversarial networks along with a differentiable ray-drop effect. We demonstrate the fidelity and diversity of our model in comparison with the point-based and image-based state-of-the-art generative models. We also showcase upsampling and restoration applications. Furthermore, we introduce a Sim2Real application for LiDAR semantic segmentation. We demonstrate that our method is effective as a realistic ray-drop simulator and outperforms state-of-the-art methods.