LiDAR-EDIT: LiDAR Data Generation by Editing the Object Layouts in Real-World Scenes

TL;DR

LiDAR-EDIT is a framework for editing real-world LiDAR scans to create diverse, realistic synthetic data with controllable object layouts, enhancing autonomous driving datasets.

Contribution

It introduces a novel method for editing real LiDAR scans to generate customizable, realistic synthetic data with preserved backgrounds and accurate geometry.

Findings

Produces realistic LiDAR scans suitable for downstream tasks

Allows full control over object placement and scene composition

Generates data with accurate geometric and label information

Abstract

We present LiDAR-EDIT, a novel paradigm for generating synthetic LiDAR data for autonomous driving. Our framework edits real-world LiDAR scans by introducing new object layouts while preserving the realism of the background environment. Compared to end-to-end frameworks that generate LiDAR point clouds from scratch, LiDAR-EDIT offers users full control over the object layout, including the number, type, and pose of objects, while keeping most of the original real-world background. Our method also provides object labels for the generated data. Compared to novel view synthesis techniques, our framework allows for the creation of counterfactual scenarios with object layouts significantly different from the original real-world scene. LiDAR-EDIT uses spherical voxelization to enforce correct LiDAR projective geometry in the generated point clouds by construction. During object removal and insertion, generative models are employed to fill the unseen background and object parts that were occluded in the original real LiDAR scans. Experimental results demonstrate that our framework produces realistic LiDAR scans with practical value for downstream tasks.