VGD: Visual Geometry Gaussian Splatting for Feed-Forward Surround-view Driving Reconstruction

TL;DR

VGD introduces a novel geometric-aware framework for surround-view autonomous driving scene reconstruction, significantly improving novel view quality and generalization by explicitly learning and leveraging geometric information.

Contribution

The paper proposes VGD, a new end-to-end framework that distills geometric priors and fuses multi-scale features for high-fidelity surround-view reconstruction.

Findings

Outperforms state-of-the-art methods on nuScenes dataset.

Achieves higher objective metrics and subjective quality.

Demonstrates scalability and robustness in various settings.

Abstract

Feed-forward surround-view autonomous driving scene reconstruction offers fast, generalizable inference ability, which faces the core challenge of ensuring generalization while elevating novel view quality. Due to the surround-view with minimal overlap regions, existing methods typically fail to ensure geometric consistency and reconstruction quality for novel views. To tackle this tension, we claim that geometric information must be learned explicitly, and the resulting features should be leveraged to guide the elevating of semantic quality in novel views. In this paper, we introduce \textbf{Visual Gaussian Driving (VGD)}, a novel feed-forward end-to-end learning framework designed to address this challenge. To achieve generalizable geometric estimation, we design a lightweight variant of the VGGT architecture to efficiently distill its geometric priors from the pre-trained VGGT to the geometry branch. Furthermore, we design a Gaussian Head that fuses multi-scale geometry tokens to predict Gaussian parameters for novel view rendering, which shares the same patch backbone as the geometry branch. Finally, we integrate multi-scale features from both geometry and Gaussian head branches to jointly supervise a semantic refinement model, optimizing rendering quality through feature-consistent learning. Experiments on nuScenes demonstrate that our approach significantly outperforms state-of-the-art methods in both objective metrics and subjective quality under various settings, which validates VGD's scalability and high-fidelity surround-view reconstruction.