fNeRF: High Quality Radiance Fields from Practical Cameras

TL;DR

fNeRF introduces a lens-aware modification to neural radiance fields, enabling more accurate and sharper 3D scene reconstructions from practical cameras with finite apertures, especially under defocus blur conditions.

Contribution

The paper proposes a lens-based ray casting model for neural radiance fields, improving reconstruction quality from real camera data by accurately modeling defocus blur.

Findings

Achieves up to 3 dB PSNR improvement on validation images.

Better matches practical camera defocus behavior than previous models.

Produces sharper reconstructions in scenes with partial occlusions.

Abstract

In recent years, the development of Neural Radiance Fields has enabled a previously unseen level of photo-realistic 3D reconstruction of scenes and objects from multi-view camera data. However, previous methods use an oversimplified pinhole camera model resulting in defocus blur being `baked' into the reconstructed radiance field. We propose a modification to the ray casting that leverages the optics of lenses to enhance scene reconstruction in the presence of defocus blur. This allows us to improve the quality of radiance field reconstructions from the measurements of a practical camera with finite aperture. We show that the proposed model matches the defocus blur behavior of practical cameras more closely than pinhole models and other approximations of defocus blur models, particularly in the presence of partial occlusions. This allows us to achieve sharper reconstructions, improving the PSNR on validation of all-in-focus images, on both synthetic and real datasets, by up to 3 dB.