FG-NeRF: Flow-GAN based Probabilistic Neural Radiance Field for Independence-Assumption-Free Uncertainty Estimation

TL;DR

This paper introduces FG-NeRF, a novel probabilistic neural radiance field that models scene density without independence assumptions, using Flow-GAN to improve uncertainty estimation and rendering accuracy.

Contribution

It presents an independence-assumption-free probabilistic NeRF leveraging Flow-GAN, combining adversarial learning and normalizing flows for better scene density modeling.

Findings

Achieves lower rendering errors on synthetic and real datasets.

Provides more reliable uncertainty estimates.

Demonstrates state-of-the-art performance in scene modeling.

Abstract

Neural radiance fields with stochasticity have garnered significant interest by enabling the sampling of plausible radiance fields and quantifying uncertainty for downstream tasks. Existing works rely on the independence assumption of points in the radiance field or the pixels in input views to obtain tractable forms of the probability density function. However, this assumption inadvertently impacts performance when dealing with intricate geometry and texture. In this work, we propose an independence-assumption-free probabilistic neural radiance field based on Flow-GAN. By combining the generative capability of adversarial learning and the powerful expressivity of normalizing flow, our method explicitly models the density-radiance distribution of the whole scene. We represent our probabilistic NeRF as a mean-shifted probabilistic residual neural model. Our model is trained without an explicit likelihood function, thereby avoiding the independence assumption. Specifically, We downsample the training images with different strides and centers to form fixed-size patches which are used to train the generator with patch-based adversarial learning. Through extensive experiments, our method demonstrates state-of-the-art performance by predicting lower rendering errors and more reliable uncertainty on both synthetic and real-world datasets.