ReinDriveGen: Reinforcement Post-Training for Out-of-Distribution Driving Scene Generation

TL;DR

ReinDriveGen is a framework that allows detailed editing of driving scenes for safety-critical scenario simulation, using reinforcement learning to improve out-of-distribution scene synthesis quality.

Contribution

It introduces a novel method combining scene editing, 3D reconstruction, and RL-based post-training to generate realistic out-of-distribution driving scenarios.

Findings

Outperforms existing methods on edited driving scenarios.

Achieves state-of-the-art in novel ego viewpoint synthesis.

Enables robust scene editing for safety-critical case simulation.

Abstract

We present ReinDriveGen, a framework that enables full controllability over dynamic driving scenes, allowing users to freely edit actor trajectories to simulate safety-critical corner cases such as front-vehicle collisions, drifting cars, vehicles spinning out of control, pedestrians jaywalking, and cyclists cutting across lanes. Our approach constructs a dynamic 3D point cloud scene from multi-frame LiDAR data, introduces a vehicle completion module to reconstruct full 360{\deg} geometry from partial observations, and renders the edited scene into 2D condition images that guide a video diffusion model to synthesize realistic driving videos. Since such edited scenarios inevitably fall outside the training distribution, we further propose an RL-based post-training strategy with a pairwise preference model and a pairwise reward mechanism, enabling robust quality improvement under out-of-distribution conditions without ground-truth supervision. Extensive experiments demonstrate that ReinDriveGen outperforms existing approaches on edited driving scenarios and achieves state-of-the-art results on novel ego viewpoint synthesis.