PNeRF: Probabilistic Neural Scene Representations for Uncertain 3D Visual Mapping

TL;DR

This paper introduces PNeRF, a probabilistic neural scene representation method that explicitly incorporates sensor and pose uncertainty during training, leading to improved 3D scene rendering and geometry accuracy in robotics applications.

Contribution

The paper proposes a novel probabilistic training approach for neural scene representations that accounts for uncertainty in sensor data and camera poses, enhancing robustness and accuracy.

Findings

Outperforms state-of-the-art methods on synthetic and real datasets.

Produces higher quality novel view renderings with limited training data.

Achieves more accurate and consistent 3D geometry.

Abstract

Recently neural scene representations have provided very impressive results for representing 3D scenes visually, however, their study and progress have mainly been limited to visualization of virtual models in computer graphics or scene reconstruction in computer vision without explicitly accounting for sensor and pose uncertainty. Using this novel scene representation in robotics applications, however, would require accounting for this uncertainty in the neural map. The aim of this paper is therefore to propose a novel method for training {\em probabilistic neural scene representations} with uncertain training data that could enable the inclusion of these representations in robotics applications. Acquiring images using cameras or depth sensors contains inherent uncertainty, and furthermore, the camera poses used for learning a 3D model are also imperfect. If these measurements are used for training without accounting for their uncertainty, then the resulting models are non-optimal, and the resulting scene representations are likely to contain artifacts such as blur and un-even geometry. In this work, the problem of uncertainty integration to the learning process is investigated by focusing on training with uncertain information in a probabilistic manner. The proposed method involves explicitly augmenting the training likelihood with an uncertainty term such that the learnt probability distribution of the network is minimized with respect to the training uncertainty. It will be shown that this leads to more accurate image rendering quality, in addition to more precise and consistent geometry. Validation has been carried out on both synthetic and real datasets showing that the proposed approach outperforms state-of-the-art methods. The results show notably that the proposed method is capable of rendering novel high-quality views even when the training data is limited.