Planar Gaussian Splatting

TL;DR

Planar Gaussian Splatting (PGS) is a neural rendering method that learns 3D scene geometry and identifies planes directly from RGB images using Gaussian primitives and hierarchical grouping, achieving state-of-the-art results without supervision.

Contribution

The paper introduces PGS, a novel neural rendering technique that models 3D scenes with Gaussian primitives and hierarchical grouping, enabling unsupervised planar reconstruction with high efficiency.

Findings

Achieves state-of-the-art 3D planar reconstruction performance.

Operates without requiring 3D plane labels or depth supervision.

Maintains robustness across different datasets and is faster than existing methods.

Abstract

This paper presents Planar Gaussian Splatting (PGS), a novel neural rendering approach to learn the 3D geometry and parse the 3D planes of a scene, directly from multiple RGB images. The PGS leverages Gaussian primitives to model the scene and employ a hierarchical Gaussian mixture approach to group them. Similar Gaussians are progressively merged probabilistically in the tree-structured Gaussian mixtures to identify distinct 3D plane instances and form the overall 3D scene geometry. In order to enable the grouping, the Gaussian primitives contain additional parameters, such as plane descriptors derived by lifting 2D masks from a general 2D segmentation model and surface normals. Experiments show that the proposed PGS achieves state-of-the-art performance in 3D planar reconstruction without requiring either 3D plane labels or depth supervision. In contrast to existing supervised methods that have limited generalizability and struggle under domain shift, PGS maintains its performance across datasets thanks to its neural rendering and scene-specific optimization mechanism, while also being significantly faster than existing optimization-based approaches.