RGE-GS: Reward-Guided Expansive Driving Scene Reconstruction via Diffusion Priors

TL;DR

RGE-GS is a novel framework that combines diffusion priors with reward-guided Gaussian integration to improve the completeness and stability of 3D scene reconstructions from driving videos, outperforming existing methods.

Contribution

The paper introduces a reward network for pattern prioritization and a differentiated training strategy to enhance scene reconstruction stability and convergence.

Findings

Achieves state-of-the-art reconstruction quality on public datasets.

Effectively integrates diffusion priors with reward guidance for stable 3D scene expansion.

Demonstrates improved convergence over baseline methods.

Abstract

A single-pass driving clip frequently results in incomplete scanning of the road structure, making reconstructed scene expanding a critical requirement for sensor simulators to effectively regress driving actions. Although contemporary 3D Gaussian Splatting (3DGS) techniques achieve remarkable reconstruction quality, their direct extension through the integration of diffusion priors often introduces cumulative physical inconsistencies and compromises training efficiency. To address these limitations, we present RGE-GS, a novel expansive reconstruction framework that synergizes diffusion-based generation with reward-guided Gaussian integration. The RGE-GS framework incorporates two key innovations: First, we propose a reward network that learns to identify and prioritize consistently generated patterns prior to reconstruction phases, thereby enabling selective retention of diffusion outputs for spatial stability. Second, during the reconstruction process, we devise a differentiated training strategy that automatically adjust Gaussian optimization progress according to scene converge metrics, which achieving better convergence than baseline methods. Extensive evaluations of publicly available datasets demonstrate that RGE-GS achieves state-of-the-art performance in reconstruction quality. Our source-code will be made publicly available at https://github.com/CN-ADLab/RGE-GS.