NeRF--: Neural Radiance Fields Without Known Camera Parameters

TL;DR

This paper introduces NeRF--, a method for novel view synthesis that jointly optimizes camera parameters and scene representation without known camera data, using a new synthetic dataset for benchmarking.

Contribution

NeRF-- removes the need for pre-computed camera parameters in NeRF training by joint optimization, and provides a new dataset and analysis for this approach.

Findings

Joint optimization can recover accurate camera parameters.

NeRF-- achieves comparable view synthesis quality without pre-known camera data.

The approach is effective across various camera motions.

Abstract

Considering the problem of novel view synthesis (NVS) from only a set of 2D images, we simplify the training process of Neural Radiance Field (NeRF) on forward-facing scenes by removing the requirement of known or pre-computed camera parameters, including both intrinsics and 6DoF poses. To this end, we propose NeRF$--$, with three contributions: First, we show that the camera parameters can be jointly optimised as learnable parameters with NeRF training, through a photometric reconstruction; Second, to benchmark the camera parameter estimation and the quality of novel view renderings, we introduce a new dataset of path-traced synthetic scenes, termed as Blender Forward-Facing Dataset (BLEFF); Third, we conduct extensive analyses to understand the training behaviours under various camera motions, and show that in most scenarios, the joint optimisation pipeline can recover accurate camera parameters and achieve comparable novel view synthesis quality as those trained with COLMAP pre-computed camera parameters. Our code and data are available at https://nerfmm.active.vision.