NeRFs: The Search for the Best 3D Representation

TL;DR

NeRFs have revolutionized 3D scene representation by modeling scenes as continuous volumetric functions using neural networks, significantly impacting view synthesis and related fields.

Contribution

This paper reviews the evolution of 3D representations leading to NeRFs and discusses recent advancements and future directions in neural scene representations.

Findings

NeRFs outperform previous 3D representations in view synthesis quality.

Numerous extensions and applications of NeRFs have emerged across computer graphics and vision.

The future of 3D representations involves integrating NeRFs with other modalities and improving efficiency.

Abstract

Neural Radiance Fields or NeRFs have become the representation of choice for problems in view synthesis or image-based rendering, as well as in many other applications across computer graphics and vision, and beyond. At their core, NeRFs describe a new representation of 3D scenes or 3D geometry. Instead of meshes, disparity maps, multiplane images or even voxel grids, they represent the scene as a continuous volume, with volumetric parameters like view-dependent radiance and volume density obtained by querying a neural network. The NeRF representation has now been widely used, with thousands of papers extending or building on it every year, multiple authors and websites providing overviews and surveys, and numerous industrial applications and startup companies. In this article, we briefly review the NeRF representation, and describe the three decades-long quest to find the best 3D representation for view synthesis and related problems, culminating in the NeRF papers. We then describe new developments in terms of NeRF representations and make some observations and insights regarding the future of 3D representations.