AutoScape: Geometry-Consistent Long-Horizon Scene Generation

TL;DR

AutoScape introduces a novel RGB-D diffusion framework for long-horizon driving scene generation, ensuring geometric consistency and high-quality video synthesis, significantly outperforming previous methods in FID and FVD metrics.

Contribution

The paper presents a new diffusion-based approach that jointly models image and depth for consistent long-horizon scene generation, with explicit geometry conditioning and warp-guided sampling.

Findings

Generates over 20 seconds of realistic driving videos.

Improves long-horizon FID by 48.6%.

Enhances FVD scores by 43.0%.

Abstract

This paper proposes AutoScape, a long-horizon driving scene generation framework. At its core is a novel RGB-D diffusion model that iteratively generates sparse, geometrically consistent keyframes, serving as reliable anchors for the scene's appearance and geometry. To maintain long-range geometric consistency, the model 1) jointly handles image and depth in a shared latent space, 2) explicitly conditions on the existing scene geometry (i.e., rendered point clouds) from previously generated keyframes, and 3) steers the sampling process with a warp-consistent guidance. Given high-quality RGB-D keyframes, a video diffusion model then interpolates between them to produce dense and coherent video frames. AutoScape generates realistic and geometrically consistent driving videos of over 20 seconds, improving the long-horizon FID and FVD scores over the prior state-of-the-art by 48.6\% and 43.0\%, respectively.