Graph-Guided Scene Reconstruction from Images with 3D Gaussian Splatting

TL;DR

This paper introduces GraphGS, a novel graph-guided framework for high-quality 3D scene reconstruction from images, addressing limitations of existing methods by leveraging camera topology and adaptive strategies for improved accuracy and efficiency.

Contribution

The paper presents a new graph-guided 3D reconstruction framework that incorporates camera topology and adaptive sampling to enhance reconstruction quality and speed.

Findings

Achieves state-of-the-art 3D reconstruction quality.

Effectively alleviates overfitting to sparse viewpoints.

Demonstrates high fidelity across multiple datasets.

Abstract

This paper investigates an open research challenge of reconstructing high-quality, large 3D open scenes from images. It is observed existing methods have various limitations, such as requiring precise camera poses for input and dense viewpoints for supervision. To perform effective and efficient 3D scene reconstruction, we propose a novel graph-guided 3D scene reconstruction framework, GraphGS. Specifically, given a set of images captured by RGB cameras on a scene, we first design a spatial prior-based scene structure estimation method. This is then used to create a camera graph that includes information about the camera topology. Further, we propose to apply the graph-guided multi-view consistency constraint and adaptive sampling strategy to the 3D Gaussian Splatting optimization process. This greatly alleviates the issue of Gaussian points overfitting to specific sparse viewpoints and expedites the 3D reconstruction process. We demonstrate GraphGS achieves high-fidelity 3D reconstruction from images, which presents state-of-the-art performance through quantitative and qualitative evaluation across multiple datasets. Project Page: https://3dagentworld.github.io/graphgs.