FewShotNeRF: Meta-Learning-based Novel View Synthesis for Rapid Scene-Specific Adaptation

TL;DR

FewShotNeRF introduces a meta-learning approach that enables rapid scene-specific adaptation of Neural Radiance Fields for novel view synthesis with limited images, significantly reducing training time.

Contribution

The paper presents a novel meta-learning framework for NeRFs that captures shared scene features, enabling fast adaptation and high-quality view synthesis from few images.

Findings

Meta-learning accelerates NeRF training for new scenes.

High-quality novel views achieved with limited multi-view images.

Robust 3D prior learned across diverse categories.

Abstract

In this paper, we address the challenge of generating novel views of real-world objects with limited multi-view images through our proposed approach, FewShotNeRF. Our method utilizes meta-learning to acquire optimal initialization, facilitating rapid adaptation of a Neural Radiance Field (NeRF) to specific scenes. The focus of our meta-learning process is on capturing shared geometry and textures within a category, embedded in the weight initialization. This approach expedites the learning process of NeRFs and leverages recent advancements in positional encodings to reduce the time required for fitting a NeRF to a scene, thereby accelerating the inner loop optimization of meta-learning. Notably, our method enables meta-learning on a large number of 3D scenes to establish a robust 3D prior for various categories. Through extensive evaluations on the Common Objects in 3D open source dataset, we empirically demonstrate the efficacy and potential of meta-learning in generating high-quality novel views of objects.