OGGSplat: Open Gaussian Growing for Generalizable Reconstruction with Expanded Field-of-View

TL;DR

OGGSplat introduces an open Gaussian growing approach that enhances 3D scene reconstruction from sparse views by expanding the field-of-view and ensuring semantic consistency through a novel inpainting and diffusion-guided optimization process.

Contribution

The paper presents OGGSplat, a novel method that enables generalizable 3D reconstruction with expanded view coverage using open Gaussians and semantic-aware inpainting, addressing limitations of existing approaches.

Findings

Effective expansion of view coverage in 3D reconstruction.

Semantic consistency maintained through diffusion-based inpainting.

Promising results on smartphone-captured images.

Abstract

Reconstructing semantic-aware 3D scenes from sparse views is a challenging yet essential research direction, driven by the demands of emerging applications such as virtual reality and embodied AI. Existing per-scene optimization methods require dense input views and incur high computational costs, while generalizable approaches often struggle to reconstruct regions outside the input view cone. In this paper, we propose OGGSplat, an open Gaussian growing method that expands the field-of-view in generalizable 3D reconstruction. Our key insight is that the semantic attributes of open Gaussians provide strong priors for image extrapolation, enabling both semantic consistency and visual plausibility. Specifically, once open Gaussians are initialized from sparse views, we introduce an RGB-semantic consistent inpainting module applied to selected rendered views. This module enforces bidirectional control between an image diffusion model and a semantic diffusion model. The inpainted regions are then lifted back into 3D space for efficient and progressive Gaussian parameter optimization. To evaluate our method, we establish a Gaussian Outpainting (GO) benchmark that assesses both semantic and generative quality of reconstructed open-vocabulary scenes. OGGSplat also demonstrates promising semantic-aware scene reconstruction capabilities when provided with two view images captured directly from a smartphone camera.