Volume Feature Rendering for Fast Neural Radiance Field Reconstruction

TL;DR

This paper introduces a novel volume feature rendering method for neural radiance fields that reduces neural network evaluations per pixel, enabling faster training and higher quality rendering with larger networks.

Contribution

It proposes rendering feature vectors first and then transforming them to pixel colors, significantly lowering computational costs and improving rendering quality.

Findings

Achieves state-of-the-art rendering quality on synthetic and real datasets.

Requires only one neural network evaluation per pixel, reducing computational complexity.

Training time is reduced to several minutes.

Abstract

Neural radiance fields (NeRFs) are able to synthesize realistic novel views from multi-view images captured from distinct positions and perspectives. In NeRF's rendering pipeline, neural networks are used to represent a scene independently or transform queried learnable feature vector of a point to the expected color or density. With the aid of geometry guides either in occupancy grids or proposal networks, the number of neural network evaluations can be reduced from hundreds to dozens in the standard volume rendering framework. Instead of rendering yielded color after neural network evaluation, we propose to render the queried feature vectors of a ray first and then transform the rendered feature vector to the final pixel color by a neural network. This fundamental change to the standard volume rendering framework requires only one single neural network evaluation to render a pixel, which substantially lowers the high computational complexity of the rendering framework attributed to a large number of neural network evaluations. Consequently, we can use a comparably larger neural network to achieve a better rendering quality while maintaining the same training and rendering time costs. Our model achieves the state-of-the-art rendering quality on both synthetic and real-world datasets while requiring a training time of several minutes.