Gaussian Set Surface Reconstruction through Per-Gaussian Optimization

TL;DR

This paper introduces GSSR, a novel method for Gaussian set surface reconstruction that optimizes Gaussian placement and normals for improved 3D scene geometry accuracy and editing capabilities.

Contribution

GSSR is the first approach to optimize Gaussian placement and normals jointly, ensuring even distribution and better geometric fidelity in 3D reconstructions.

Findings

Significantly improved geometric accuracy in Gaussian placement.

Enhanced scene editing and novel view synthesis capabilities.

Effective regularization and reinitialization techniques for cleaner distributions.

Abstract

3D Gaussian Splatting (3DGS) effectively synthesizes novel views through its flexible representation, yet fails to accurately reconstruct scene geometry. While modern variants like PGSR introduce additional losses to ensure proper depth and normal maps through Gaussian fusion, they still neglect individual placement optimization. This results in unevenly distributed Gaussians that deviate from the latent surface, complicating both reconstruction refinement and scene editing. Motivated by pioneering work on Point Set Surfaces, we propose Gaussian Set Surface Reconstruction (GSSR), a method designed to distribute Gaussians evenly along the latent surface while aligning their dominant normals with the surface normal. GSSR enforces fine-grained geometric alignment through a combination of pixel-level and Gaussian-level single-view normal consistency and multi-view photometric consistency, optimizing both local and global perspectives. To further refine the representation, we introduce an opacity regularization loss to eliminate redundant Gaussians and apply periodic depth- and normal-guided Gaussian reinitialization for a cleaner, more uniform spatial distribution. Our reconstruction results demonstrate significantly improved geometric precision in Gaussian placement, enabling intuitive scene editing and efficient generation of novel Gaussian-based 3D environments. Extensive experiments validate GSSR's effectiveness, showing enhanced geometric accuracy while preserving high-quality rendering performance.