SG-LDM: Semantic-Guided LiDAR Generation via Latent-Aligned Diffusion

TL;DR

SG-LDM introduces a semantic-guided diffusion model for high-fidelity lidar point cloud synthesis and cross-domain translation, significantly improving data augmentation and perception tasks.

Contribution

The paper presents the first semantic-guided diffusion model for lidar generation and a diffusion-based lidar translation framework for domain adaptation.

Findings

Outperforms existing lidar diffusion models in fidelity and accuracy.

Enables effective cross-domain lidar translation for data augmentation.

Enhances downstream lidar segmentation performance.

Abstract

Lidar point cloud synthesis based on generative models offers a promising solution to augment deep learning pipelines, particularly when real-world data is scarce or lacks diversity. By enabling flexible object manipulation, this synthesis approach can significantly enrich training datasets and enhance discriminative models. However, existing methods focus on unconditional lidar point cloud generation, overlooking their potential for real-world applications. In this paper, we propose SG-LDM, a Semantic-Guided Lidar Diffusion Model that employs latent alignment to enable robust semantic-to-lidar synthesis. By directly operating in the native lidar space and leveraging explicit semantic conditioning, SG-LDM achieves state-of-the-art performance in generating high-fidelity lidar point clouds guided by semantic labels. Moreover, we propose the first diffusion-based lidar translation framework based on SG-LDM, which enables cross-domain translation as a domain adaptation strategy to enhance downstream perception performance. Systematic experiments demonstrate that SG-LDM significantly outperforms existing lidar diffusion models and the proposed lidar translation framework further improves data augmentation performance in the downstream lidar segmentation task.