L3DR: 3D-aware LiDAR Diffusion and Rectification

TL;DR

L3DR introduces a 3D-aware diffusion and rectification framework for LiDAR data that improves geometric realism and artifact correction, outperforming existing methods across multiple benchmarks.

Contribution

The paper proposes a novel 3D residual regression network and a Welsch Loss for LiDAR artifact rectification, enhancing 3D geometry realism in diffusion models.

Findings

Achieves state-of-the-art results on KITTI, nuScenes, Waymo datasets.

Effectively rectifies RV artifacts and improves local geometry.

Applicable to various LiDAR diffusion models with low computational cost.

Abstract

Range-view (RV) based LiDAR diffusion has recently made huge strides towards 2D photo-realism. However, it neglects 3D geometry realism and often generates various RV artifacts such as depth bleeding and wavy surfaces. We design L3DR, a 3D-aware LiDAR Diffusion and Rectification framework that can regress and cancel RV artifacts in 3D space and restore local geometry accurately. Our theoretical and empirical analysis reveals that 3D models are inherently superior to 2D models in generating sharp and authentic boundaries. Leveraging such analysis, we design a 3D residual regression network that rectifies RV artifacts and achieves superb geometry realism by predicting point-level offsets in 3D space. On top of that, we design a Welsch Loss that helps focus on local geometry and ignore anomalous regions effectively. Extensive experiments over multiple benchmarks including KITTI, KITTI360, nuScenes and Waymo show that the proposed L3DR achieves state-of-the-art generation and superior geometry-realism consistently. In addition, L3DR is generally applicable to different LiDAR diffusion models with little computational overhead.