Few-shot NeRF by Adaptive Rendering Loss Regularization

TL;DR

This paper introduces AR-NeRF, a novel method that aligns frequency regularization with rendering loss in few-shot NeRF, improving the synthesis of high-quality novel views from sparse inputs.

Contribution

We propose an adaptive rendering loss regularization technique that addresses the inconsistency between frequency regularization and rendering loss in few-shot NeRF.

Findings

AR-NeRF achieves state-of-the-art results on multiple datasets.

The method improves global structure learning in early training.

Adaptive loss weighting enhances local detail synthesis.

Abstract

Novel view synthesis with sparse inputs poses great challenges to Neural Radiance Field (NeRF). Recent works demonstrate that the frequency regularization of Positional Encoding (PE) can achieve promising results for few-shot NeRF. In this work, we reveal that there exists an inconsistency between the frequency regularization of PE and rendering loss. This prevents few-shot NeRF from synthesizing higher-quality novel views. To mitigate this inconsistency, we propose Adaptive Rendering loss regularization for few-shot NeRF, dubbed AR-NeRF. Specifically, we present a two-phase rendering supervision and an adaptive rendering loss weight learning strategy to align the frequency relationship between PE and 2D-pixel supervision. In this way, AR-NeRF can learn global structures better in the early training phase and adaptively learn local details throughout the training process. Extensive experiments show that our AR-NeRF achieves state-of-the-art performance on different datasets, including object-level and complex scenes.