Geo-EVS: Geometry-Conditioned Extrapolative View Synthesis for Autonomous Driving

TL;DR

Geo-EVS introduces a geometry-conditioned framework for extrapolative view synthesis in autonomous driving, enhancing accuracy and robustness in sparse and out-of-trajectory scenarios.

Contribution

It presents a novel geometry-aware reprojection method and artifact-guided latent diffusion to improve extrapolative view synthesis under sparse supervision.

Findings

Improves sparse-view synthesis quality on Waymo dataset.

Enhances geometric accuracy in high-angle and low-coverage settings.

Boosts downstream 3D detection performance.

Abstract

Extrapolative novel view synthesis can reduce camera-rig dependency in autonomous driving by generating standardized virtual views from heterogeneous sensors. Existing methods degrade outside recorded trajectories because extrapolated poses provide weak geometric support and no dense target-view supervision. The key is to explicitly expose the model to out-of-trajectory condition defects during training. We propose Geo-EVS, a geometry-conditioned framework under sparse supervision. Geo-EVS has two components. Geometry-Aware Reprojection (GAR) uses fine-tuned VGGT to reconstruct colored point clouds and reproject them to observed and virtual target poses, producing geometric condition maps. This design unifies the reprojection path between training and inference. Artifact-Guided Latent Diffusion (AGLD) injects reprojection-derived artifact masks during training so the model learns to recover structure under missing support. For evaluation, we use a LiDAR-Projected Sparse-Reference (LPSR) protocol when dense extrapolated-view ground truth is unavailable. On Waymo, Geo-EVS improves sparse-view synthesis quality and geometric accuracy, especially in high-angle and low-coverage settings. It also improves downstream 3D detection.