Mirage: One-Step Video Diffusion for Photorealistic and Coherent Asset Editing in Driving Scenes

TL;DR

Mirage is a novel one-step video diffusion model that enables photorealistic, temporally coherent asset editing in driving scenes, addressing fidelity and consistency challenges in autonomous driving data augmentation.

Contribution

We introduce Mirage, a one-step video diffusion approach with a new latent injection and data alignment strategy for improved asset editing in driving videos.

Findings

Achieves high realism and temporal coherence in editing scenarios

Outperforms existing methods in visual fidelity and consistency

Generalizes well to other video translation tasks

Abstract

Vision-centric autonomous driving systems rely on diverse and scalable training data to achieve robust performance. While video object editing offers a promising path for data augmentation, existing methods often struggle to maintain both high visual fidelity and temporal coherence. In this work, we propose \textbf{Mirage}, a one-step video diffusion model for photorealistic and coherent asset editing in driving scenes. Mirage builds upon a text-to-video diffusion prior to ensure temporal consistency across frames. However, 3D causal variational autoencoders often suffer from degraded spatial fidelity due to compression, and directly passing 3D encoder features to decoder layers breaks temporal causality. To address this, we inject temporally agnostic latents from a pretrained 2D encoder into the 3D decoder to restore detail while preserving causal structures. Furthermore, because scene objects and inserted assets are optimized under different objectives, their Gaussians exhibit a distribution mismatch that leads to pose misalignment. To mitigate this, we introduce a two-stage data alignment strategy combining coarse 3D alignment and fine 2D refinement, thereby improving alignment and providing cleaner supervision. Extensive experiments demonstrate that Mirage achieves high realism and temporal consistency across diverse editing scenarios. Beyond asset editing, Mirage can also generalize to other video-to-video translation tasks, serving as a reliable baseline for future research. Our code is available at https://github.com/wm-research/mirage.