LiDAR Data Synthesis with Denoising Diffusion Probabilistic Models

TL;DR

This paper introduces R2DM, a novel denoising diffusion probabilistic model for generating high-fidelity 3D LiDAR point clouds from image representations, improving over existing methods in fidelity and stability.

Contribution

The paper presents R2DM, the first diffusion-based generative model tailored for LiDAR data, with a comprehensive analysis of data representation and a new LiDAR completion pipeline.

Findings

R2DM outperforms existing methods on KITTI datasets.

The model achieves high-fidelity and diverse 3D scene generation.

Effective training strategies for diffusion models in LiDAR domain.

Abstract

Generative modeling of 3D LiDAR data is an emerging task with promising applications for autonomous mobile robots, such as scalable simulation, scene manipulation, and sparse-to-dense completion of LiDAR point clouds. While existing approaches have demonstrated the feasibility of image-based LiDAR data generation using deep generative models, they still struggle with fidelity and training stability. In this work, we present R2DM, a novel generative model for LiDAR data that can generate diverse and high-fidelity 3D scene point clouds based on the image representation of range and reflectance intensity. Our method is built upon denoising diffusion probabilistic models (DDPMs), which have shown impressive results among generative model frameworks in recent years. To effectively train DDPMs in the LiDAR domain, we first conduct an in-depth analysis of data representation, loss functions, and spatial inductive biases. Leveraging our R2DM model, we also introduce a flexible LiDAR completion pipeline based on the powerful capabilities of DDPMs. We demonstrate that our method surpasses existing methods in generating tasks on the KITTI-360 and KITTI-Raw datasets, as well as in the completion task on the KITTI-360 dataset. Our project page can be found at https://kazuto1011.github.io/r2dm.