PCGen: Point Cloud Generator for LiDAR Simulation

TL;DR

This paper introduces PCGen, a LiDAR point cloud simulator that improves data realism and includes deformable VRUs, enabling better training for perception systems with reduced real data reliance.

Contribution

The paper presents FPA raycasting and surrogate models to generate more realistic point clouds and extends simulation to deformable VRUs, addressing key limitations of prior methods.

Findings

Surrogate model effectively mimics LiDAR sensor properties.

Simulation-trained models achieve comparable performance to real data-trained models.

Proposed methods reduce domain gap between simulated and real point clouds.

Abstract

Data is a fundamental building block for LiDAR perception systems. Unfortunately, real-world data collection and annotation is extremely costly & laborious. Recently, real data based LiDAR simulators have shown tremendous potential to complement real data, due to their scalability and high-fidelity compared to graphics engine based methods. Before simulation can be deployed in the real-world, two shortcomings need to be addressed. First, existing methods usually generate data which are more noisy and complete than the real point clouds, due to 3D reconstruction error and pure geometry-based raycasting method. Second, prior works on simulation for object detection focus solely on rigid objects, like cars, but VRUs, like pedestrians, are important road participants. To tackle the first challenge, we propose FPA raycasting and surrogate model raydrop. FPA enables the simulation of both point cloud coordinates and sensor features, while taking into account reconstruction noise. The ray-wise surrogate raydrop model mimics the physical properties of LiDAR's laser receiver to determine whether a simulated point would be recorded by a real LiDAR. With minimal training data, the surrogate model can generalize to different geographies and scenes, closing the domain gap between raycasted and real point clouds. To tackle the simulation of deformable VRU simulation, we employ SMPL dataset to provide a pedestrian simulation baseline and compare the domain gap between CAD and reconstructed objects. Applying our pipeline to perform novel sensor synthesis, results show that object detection models trained by simulation data can achieve similar result as the real data trained model.