LighthouseGS: Indoor Structure-aware 3D Gaussian Splatting for Panorama-Style Mobile Captures

TL;DR

LighthouseGS is a novel view synthesis framework that uses indoor geometric priors and planar structures to improve 3D scene reconstruction and rendering from simple mobile panorama captures.

Contribution

It introduces a new initialization method called plane scaffold assembly and geometric corrections to enhance 3D reconstruction stability in indoor scenes.

Findings

Outperforms state-of-the-art methods in indoor scene rendering

Enables realistic panoramic view synthesis from mobile captures

Improves 3D geometry accuracy using planar structure priors

Abstract

We introduce LighthouseGS, a practical novel view synthesis framework based on 3D Gaussian Splatting that utilizes simple panorama-style captures from a single mobile device. While convenient, this rotation-dominant motion and narrow baseline make accurate camera pose and 3D point estimation challenging, especially in textureless indoor scenes. To address these challenges, LighthouseGS leverages rough geometric priors, such as mobile device camera poses and monocular depth estimation, and utilizes indoor planar structures. Specifically, we propose a new initialization method called plane scaffold assembly to generate consistent 3D points on these structures, followed by a stable pruning strategy to enhance geometry and optimization stability. Additionally, we present geometric and photometric corrections to resolve inconsistencies from motion drift and auto-exposure in mobile devices. Tested on real and synthetic indoor scenes, LighthouseGS delivers photorealistic rendering, outperforming state-of-the-art methods and enabling applications like panoramic view synthesis and object placement. Project page: https://vision3d-lab.github.io/lighthousegs/