RefiNeRF: Modelling dynamic neural radiance fields with inconsistent or missing camera parameters

TL;DR

RefiNeRF introduces a method to learn camera parameters directly from unposed images in dynamic scenes, enhancing neural radiance fields for novel view synthesis without relying on traditional calibration techniques.

Contribution

The paper presents a novel approach to estimate camera parameters directly from data, integrating seamlessly with NeRF architectures, and outperforming traditional SfM and MVS methods.

Findings

Outperforms traditional SfM and MVS in accuracy.

Effectively models static and dynamic scenes.

Provides a flexible, data-driven camera calibration method.

Abstract

Novel view synthesis (NVS) is a challenging task in computer vision that involves synthesizing new views of a scene from a limited set of input images. Neural Radiance Fields (NeRF) have emerged as a powerful approach to address this problem, but they require accurate knowledge of camera \textit{intrinsic} and \textit{extrinsic} parameters. Traditionally, structure-from-motion (SfM) and multi-view stereo (MVS) approaches have been used to extract camera parameters, but these methods can be unreliable and may fail in certain cases. In this paper, we propose a novel technique that leverages unposed images from dynamic datasets, such as the NVIDIA dynamic scenes dataset, to learn camera parameters directly from data. Our approach is highly extensible and can be integrated into existing NeRF architectures with minimal modifications. We demonstrate the effectiveness of our method on a variety of static and dynamic scenes and show that it outperforms traditional SfM and MVS approaches. The code for our method is publicly available at \href{https://github.com/redacted/refinerf}{https://github.com/redacted/refinerf}. Our approach offers a promising new direction for improving the accuracy and robustness of NVS using NeRF, and we anticipate that it will be a valuable tool for a wide range of applications in computer vision and graphics.