DriveLiDAR4D: Sequential and Controllable LiDAR Scene Generation for Autonomous Driving

TL;DR

DriveLiDAR4D introduces a novel, controllable, and sequential LiDAR scene generation method that produces realistic, temporally consistent 3D point clouds for autonomous driving, surpassing existing state-of-the-art techniques.

Contribution

It is the first to enable end-to-end sequential LiDAR scene generation with full scene manipulation capabilities.

Findings

Achieved an FRD score of 743.13 on nuScenes

Achieved an FVD score of 16.96 on nuScenes

Surpassed SOTA by 37.2% in FRD and 24.1% in FVD

Abstract

The generation of realistic LiDAR point clouds plays a crucial role in the development and evaluation of autonomous driving systems. Although recent methods for 3D LiDAR point cloud generation have shown significant improvements, they still face notable limitations, including the lack of sequential generation capabilities and the inability to produce accurately positioned foreground objects and realistic backgrounds. These shortcomings hinder their practical applicability. In this paper, we introduce DriveLiDAR4D, a novel LiDAR generation pipeline consisting of multimodal conditions and a novel sequential noise prediction model LiDAR4DNet, capable of producing temporally consistent LiDAR scenes with highly controllable foreground objects and realistic backgrounds. To the best of our knowledge, this is the first work to address the sequential generation of LiDAR scenes with full scene manipulation capability in an end-to-end manner. We evaluated DriveLiDAR4D on the nuScenes and KITTI datasets, where we achieved an FRD score of 743.13 and an FVD score of 16.96 on the nuScenes dataset, surpassing the current state-of-the-art (SOTA) method, UniScene, with an performance boost of 37.2% in FRD and 24.1% in FVD, respectively.