LaGen: Towards Autoregressive LiDAR Scene Generation

TL;DR

LaGen is a novel autoregressive framework for long-horizon LiDAR scene generation that effectively uses single-frame input and bounding box conditions to produce high-fidelity 4D point clouds, advancing autonomous driving simulation.

Contribution

Introduces LaGen, the first autoregressive model for frame-by-frame LiDAR scene generation, with modules for scene decoupling and noise modulation to improve interactivity and long-term accuracy.

Findings

Outperforms existing LiDAR generation models on nuScenes

Effective in long-horizon scene generation with high fidelity

Enhances object-level interactivity in LiDAR scene synthesis

Abstract

Generative world models for autonomous driving (AD) have become a trending topic. Unlike the widely studied image modality, in this work we explore generative world models for LiDAR data. Existing generation methods for LiDAR data only support single frame generation, while existing prediction approaches require multiple frames of historical input and can only deterministically predict multiple frames at once, lacking interactivity. Both paradigms fail to support long-horizon interactive generation. To this end, we introduce LaGen, which to the best of our knowledge is the first framework capable of frame-by-frame autoregressive generation of long-horizon LiDAR scenes. LaGen is able to take a single-frame LiDAR input as a starting point and effectively utilize bounding box information as conditions to generate high-fidelity 4D scene point clouds. In addition, we introduce a scene decoupling estimation module to enhance the model's interactive generation capability for object-level content, as well as a noise modulation module to mitigate error accumulation during long-horizon generation. We construct a protocol based on nuScenes for evaluating long-horizon LiDAR scene generation. Experimental results comprehensively demonstrate LaGen outperforms state-of-the-art LiDAR generation and prediction models, especially on the later frames.