MultiPlaneNeRF: Neural Radiance Field with Non-Trainable Representation

TL;DR

MultiPlaneNeRF introduces a non-trainable, projection-based approach to neural radiance fields that generalizes across objects and reduces training time, enabling efficient 3D reconstruction from 2D images.

Contribution

The paper proposes MultiPlaneNeRF, a novel model that uses non-trainable 2D projections for efficient, generalizable 3D object representation without per-object training.

Findings

Achieves comparable results to state-of-the-art NeRF models in view synthesis.

Demonstrates strong generalization across multiple objects without retraining.

The decoder can be integrated into large generative models like GANs.

Abstract

NeRF is a popular model that efficiently represents 3D objects from 2D images. However, vanilla NeRF has some important limitations. NeRF must be trained on each object separately. The training time is long since we encode the object's shape and color in neural network weights. Moreover, NeRF does not generalize well to unseen data. In this paper, we present MultiPlaneNeRF -- a model that simultaneously solves the above problems. Our model works directly on 2D images. We project 3D points on 2D images to produce non-trainable representations. The projection step is not parametrized and a very shallow decoder can efficiently process the representation. Furthermore, we can train MultiPlaneNeRF on a large data set and force our implicit decoder to generalize across many objects. Consequently, we can only replace the 2D images (without additional training) to produce a NeRF representation of the new object. In the experimental section, we demonstrate that MultiPlaneNeRF achieves results comparable to state-of-the-art models for synthesizing new views and has generalization properties. Additionally, MultiPlane decoder can be used as a component in large generative models like GANs.