Manifold Sampling for Differentiable Uncertainty in Radiance Fields

TL;DR

This paper introduces a method for learning Gaussian radiance fields with explicit uncertainty estimates, enabling better scene understanding and optimized view planning with minimal additional computational cost.

Contribution

It presents a novel approach modeling uncertainties as a low-dimensional manifold in radiance field parameters, facilitating differentiable and efficient uncertainty estimation.

Findings

Achieves state-of-the-art results in next-best-view planning.

Effectively models uncertainties as a low-dimensional manifold.

Enhances radiance field relighting quality through optimized view planning.

Abstract

Radiance fields are powerful and, hence, popular models for representing the appearance of complex scenes. Yet, constructing them based on image observations gives rise to ambiguities and uncertainties. We propose a versatile approach for learning Gaussian radiance fields with explicit and fine-grained uncertainty estimates that impose only little additional cost compared to uncertainty-agnostic training. Our key observation is that uncertainties can be modeled as a low-dimensional manifold in the space of radiance field parameters that is highly amenable to Monte Carlo sampling. Importantly, our uncertainties are differentiable and, thus, allow for gradient-based optimization of subsequent captures that optimally reduce ambiguities. We demonstrate state-of-the-art performance on next-best-view planning tasks, including high-dimensional illumination planning for optimal radiance field relighting quality.