FrugalNeRF: Fast Convergence for Extreme Few-shot Novel View Synthesis without Learned Priors

TL;DR

FrugalNeRF introduces a fast, efficient few-shot NeRF framework that uses cross-scale geometric adaptation and weight-sharing voxels to improve scene reconstruction without relying heavily on learned priors.

Contribution

It proposes a novel cross-scale geometric adaptation scheme and voxel weight-sharing approach that enable rapid convergence in few-shot NeRF without external priors.

Findings

Outperforms existing few-shot NeRF methods in quality.

Reduces training time significantly.

Effective with or without pre-trained priors.

Abstract

Neural Radiance Fields (NeRF) face significant challenges in extreme few-shot scenarios, primarily due to overfitting and long training times. Existing methods, such as FreeNeRF and SparseNeRF, use frequency regularization or pre-trained priors but struggle with complex scheduling and bias. We introduce FrugalNeRF, a novel few-shot NeRF framework that leverages weight-sharing voxels across multiple scales to efficiently represent scene details. Our key contribution is a cross-scale geometric adaptation scheme that selects pseudo ground truth depth based on reprojection errors across scales. This guides training without relying on externally learned priors, enabling full utilization of the training data. It can also integrate pre-trained priors, enhancing quality without slowing convergence. Experiments on LLFF, DTU, and RealEstate-10K show that FrugalNeRF outperforms other few-shot NeRF methods while significantly reducing training time, making it a practical solution for efficient and accurate 3D scene reconstruction.