Modeling Uncertainty in 3D Gaussian Splatting through Continuous Semantic Splatting

TL;DR

This paper introduces a probabilistic approach to semantic mapping in 3D Gaussian Splatting, enabling uncertainty quantification and improved safety for robotic scene understanding.

Contribution

It extends 3D Gaussian Splatting from voxel-based to ellipsoid-based semantic mapping with probabilistic updates and rasterization, incorporating uncertainty estimation.

Findings

Effective semantic updates with uncertainty quantification.

Improved scene understanding through probabilistic rasterization.

Enhanced safety in robotic applications.

Abstract

In this paper, we present a novel algorithm for probabilistically updating and rasterizing semantic maps within 3D Gaussian Splatting (3D-GS). Although previous methods have introduced algorithms which learn to rasterize features in 3D-GS for enhanced scene understanding, 3D-GS can fail without warning which presents a challenge for safety-critical robotic applications. To address this gap, we propose a method which advances the literature of continuous semantic mapping from voxels to ellipsoids, combining the precise structure of 3D-GS with the ability to quantify uncertainty of probabilistic robotic maps. Given a set of images, our algorithm performs a probabilistic semantic update directly on the 3D ellipsoids to obtain an expectation and variance through the use of conjugate priors. We also propose a probabilistic rasterization which returns per-pixel segmentation predictions with quantifiable uncertainty. We compare our method with similar probabilistic voxel-based methods to verify our extension to 3D ellipsoids, and perform ablation studies on uncertainty quantification and temporal smoothing.