Self-NeRF: A Self-Training Pipeline for Few-Shot Neural Radiance Fields

TL;DR

Self-NeRF introduces a self-training pipeline for neural radiance fields that effectively improves novel view synthesis in few-shot scenarios by iteratively refining the model with pseudo-views and uncertainty modeling.

Contribution

The paper presents a novel self-evolved NeRF framework that refines radiance fields with minimal views without extra priors, using iterative pseudo-view labeling and uncertainty-aware embeddings.

Findings

Outperforms existing methods in limited data settings.

Robust to input uncertainty and artifacts.

Effective pseudo-view utilization with regularization techniques.

Abstract

Recently, Neural Radiance Fields (NeRF) have emerged as a potent method for synthesizing novel views from a dense set of images. Despite its impressive performance, NeRF is plagued by its necessity for numerous calibrated views and its accuracy diminishes significantly in a few-shot setting. To address this challenge, we propose Self-NeRF, a self-evolved NeRF that iteratively refines the radiance fields with very few number of input views, without incorporating additional priors. Basically, we train our model under the supervision of reference and unseen views simultaneously in an iterative procedure. In each iteration, we label unseen views with the predicted colors or warped pixels generated by the model from the preceding iteration. However, these expanded pseudo-views are afflicted by imprecision in color and warping artifacts, which degrades the performance of NeRF. To alleviate this issue, we construct an uncertainty-aware NeRF with specialized embeddings. Some techniques such as cone entropy regularization are further utilized to leverage the pseudo-views in the most efficient manner. Through experiments under various settings, we verified that our Self-NeRF is robust to input with uncertainty and surpasses existing methods when trained on limited training data.