Reducing Shape-Radiance Ambiguity in Radiance Fields with a Closed-Form Color Estimation Method

TL;DR

This paper introduces a novel, scene-adaptive method to reduce shape-radiance ambiguity in neural radiance fields by estimating the color field directly from the density field in a closed form, improving the quality of 3D scene reconstructions.

Contribution

It proposes a closed-form color estimation method based solely on the density field to better decouple shape and radiance in NeRF, addressing limitations of prior scene-agnostic regularizations.

Findings

Improves the quality of density fields both qualitatively and quantitatively.

Provides a more adaptive regularization method guided by photometric loss.

Enhances the decoupling of shape and radiance in NeRF reconstructions.

Abstract

Neural radiance field (NeRF) enables the synthesis of cutting-edge realistic novel view images of a 3D scene. It includes density and color fields to model the shape and radiance of a scene, respectively. Supervised by the photometric loss in an end-to-end training manner, NeRF inherently suffers from the shape-radiance ambiguity problem, i.e., it can perfectly fit training views but does not guarantee decoupling the two fields correctly. To deal with this issue, existing works have incorporated prior knowledge to provide an independent supervision signal for the density field, including total variation loss, sparsity loss, distortion loss, etc. These losses are based on general assumptions about the density field, e.g., it should be smooth, sparse, or compact, which are not adaptive to a specific scene. In this paper, we propose a more adaptive method to reduce the shape-radiance ambiguity. The key is a rendering method that is only based on the density field. Specifically, we first estimate the color field based on the density field and posed images in a closed form. Then NeRF's rendering process can proceed. We address the problems in estimating the color field, including occlusion and non-uniformly distributed views. Afterward, it is applied to regularize NeRF's density field. As our regularization is guided by photometric loss, it is more adaptive compared to existing ones. Experimental results show that our method improves the density field of NeRF both qualitatively and quantitatively. Our code is available at https://github.com/qihangGH/Closed-form-color-field.