PRIMU: Uncertainty Estimation for Novel Views in Gaussian Splatting from Primitive-Based Representations of Error and Coverage

TL;DR

PRIMU introduces a primitive-level uncertainty estimation framework for Gaussian Splatting that improves accuracy, interpretability, and generalization in novel view synthesis, especially in safety-critical applications.

Contribution

PRIMU provides a novel primitive-based uncertainty estimation method that captures error and coverage, enabling better uncertainty prediction and active view selection in Gaussian Splatting.

Findings

PRIMU's uncertainty estimates strongly correlate with true errors.

PRIMU outperforms state-of-the-art methods in depth uncertainty estimation.

PRIMU generalizes to unseen scenes without additional holdout data.

Abstract

We introduce Primitive-based Representations of Uncertainty (PRIMU), a post-hoc uncertainty estimation (UE) framework for Gaussian Splatting (GS). Reliable UE is essential for deploying GS in safety-critical domains such as robotics and medicine. Existing approaches typically estimate Gaussian-primitive variances and rely on the rendering process to obtain pixel-wise uncertainties. In contrast, we construct primitive-level representations of error and visibility/coverage from training views, capturing interpretable uncertainty information. These representations are obtained by projecting view-dependent training errors and coverage statistics onto the primitives. Uncertainties for novel views are inferred by rendering these primitive-level representations, producing uncertainty feature maps, which are aggregate through pixel-wise regression on holdout data. We analyze combinations of uncertainty feature maps and regression models to understand how their interactions affect prediction accuracy and generalization. PRIMU also enables an effective active view selection strategy by directly leveraging these uncertainty feature maps. Additionally, we study the effect of separating splatting into foreground and background regions. Our estimates show strong correlations with true errors, outperforming state-of-the-art methods, especially for depth UE and foreground objects. Finally, our regression models show generalization capabilities to unseen scenes, enabling UE without additional holdout data.