A Probability-guided Sampler for Neural Implicit Surface Rendering

TL;DR

This paper introduces a probability-guided sampling method and a new surface reconstruction loss for neural implicit surface rendering, leading to more accurate 3D reconstructions and better image quality by focusing on regions of interest.

Contribution

It proposes a novel probability density function-based sampling strategy and a comprehensive surface reconstruction loss for neural implicit surface rendering.

Findings

Enhanced 3D reconstruction accuracy

Improved rendering quality in regions of interest

Better handling of near-to-surface and empty space regions

Abstract

Several variants of Neural Radiance Fields (NeRFs) have significantly improved the accuracy of synthesized images and surface reconstruction of 3D scenes/objects. In all of these methods, a key characteristic is that none can train the neural network with every possible input data, specifically, every pixel and potential 3D point along the projection rays due to scalability issues. While vanilla NeRFs uniformly sample both the image pixels and 3D points along the projection rays, some variants focus only on guiding the sampling of the 3D points along the projection rays. In this paper, we leverage the implicit surface representation of the foreground scene and model a probability density function in a 3D image projection space to achieve a more targeted sampling of the rays toward regions of interest, resulting in improved rendering. Additionally, a new surface reconstruction loss is proposed for improved performance. This new loss fully explores the proposed 3D image projection space model and incorporates near-to-surface and empty space components. By integrating our novel sampling strategy and novel loss into current state-of-the-art neural implicit surface renderers, we achieve more accurate and detailed 3D reconstructions and improved image rendering, especially for the regions of interest in any given scene.