DriveExplorer: Images-Only Decoupled 4D Reconstruction with Progressive Restoration for Driving View Extrapolation

TL;DR

DriveExplorer introduces a novel image-only 4D scene reconstruction method for driving view extrapolation, utilizing progressive diffusion refinement to generate high-quality novel views without relying on expensive sensors.

Contribution

It proposes a decoupled 4D reconstruction framework that uses only images and camera poses, integrating a Gaussian model with diffusion refinement for view extrapolation.

Findings

Produces higher-quality extrapolated images than baselines.

Operates without LiDAR or extensive annotations.

Effectively refines scene representations through diffusion models.

Abstract

This paper presents an effective solution for view extrapolation in autonomous driving scenarios. Recent approaches focus on generating shifted novel view images from given viewpoints using diffusion models. However, these methods heavily rely on priors such as LiDAR point clouds, 3D bounding boxes, and lane annotations, which demand expensive sensors or labor-intensive labeling, limiting applicability in real-world deployment. In this work, with only images and optional camera poses, we first estimate a global static point cloud and per-frame dynamic point clouds, fusing them into a unified representation. We then employ a deformable 4D Gaussian framework to reconstruct the scene. The initially trained 4D Gaussian model renders degraded and pseudo-images to train a video diffusion model. Subsequently, progressively shifted Gaussian renderings are iteratively refined by the diffusion model,and the enhanced results are incorporated back as training data for 4DGS. This process continues until extrapolation reaches the target viewpoints. Compared with baselines, our method produces higher-quality images at novel extrapolated viewpoints.