SymDrive: Realistic and Controllable Driving Simulator via Symmetric Auto-regressive Online Restoration

TL;DR

SymDrive is a diffusion-based framework that enables photorealistic 3D scene rendering and editing for autonomous driving simulation, addressing view synthesis and scene manipulation challenges with a novel symmetric auto-regressive restoration approach.

Contribution

It introduces a symmetric auto-regressive online restoration paradigm for high-quality rendering and scene editing in autonomous driving simulation.

Findings

Achieves state-of-the-art novel-view synthesis quality.

Enables realistic 3D vehicle insertion with seamless lighting.

Demonstrates superior performance in scene editing tasks.

Abstract

High-fidelity and controllable 3D simulation is essential for addressing the long-tail data scarcity in Autonomous Driving (AD), yet existing methods struggle to simultaneously achieve photorealistic rendering and interactive traffic editing. Current approaches often falter in large-angle novel view synthesis and suffer from geometric or lighting artifacts during asset manipulation. To address these challenges, we propose SymDrive, a unified diffusion-based framework capable of joint high-quality rendering and scene editing. We introduce a Symmetric Auto-regressive Online Restoration paradigm, which constructs paired symmetric views to recover fine-grained details via a ground-truth-guided dual-view formulation and utilizes an auto-regressive strategy for consistent lateral view generation. Furthermore, we leverage this restoration capability to enable a training-free harmonization mechanism, treating vehicle insertion as context-aware inpainting to ensure seamless lighting and shadow consistency. Extensive experiments demonstrate that SymDrive achieves state-of-the-art performance in both novel-view enhancement and realistic 3D vehicle insertion.