$\Delta$-NeRF: Incremental Refinement of Neural Radiance Fields through Residual Control and Knowledge Transfer

TL;DR

$ ext{Delta-NeRF}$ introduces a modular residual framework for incremental neural radiance field refinement, enabling efficient updates without retraining from scratch and reducing data needs, with applications in satellite imagery.

Contribution

The paper presents $ ext{Delta-NeRF}$, a novel incremental refinement method for NeRFs using residual control, uncertainty gating, and knowledge distillation, addressing catastrophic forgetting and data efficiency.

Findings

Achieves comparable performance to joint training with less training time.

Reduces training data by up to 47% while maintaining quality.

Outperforms naive fine-tuning with up to 43.5% PSNR improvement.

Abstract

Neural Radiance Fields (NeRFs) have demonstrated remarkable capabilities in 3D reconstruction and novel view synthesis. However, most existing NeRF frameworks require complete retraining when new views are introduced incrementally, limiting their applicability in domains where data arrives sequentially. This limitation is particularly problematic in satellite-based terrain analysis, where regions are repeatedly observed over time. Incremental refinement of NeRFs remains underexplored, and naive approaches suffer from catastrophic forgetting when past data is unavailable. We propose $\Delta$-NeRF, a unique modular residual framework for incremental NeRF refinement. $\Delta$-NeRF introduces several novel techniques including: (1) a residual controller that injects per-layer corrections into a frozen base NeRF, enabling refinement without access to past data; (2) an uncertainty-aware gating mechanism that prevents overcorrection by adaptively combining base and refined predictions; and (3) a view selection strategy that reduces training data by up to 47\% while maintaining performance. Additionally, we employ knowledge distillation to compress the enhanced model into a compact student network (20\% of original size). Experiments on satellite imagery demonstrate that $\Delta$-NeRF achieves performance comparable to joint training while reducing training time by 30-42\%. $\Delta$-NeRF consistently outperforms existing baselines, achieving an improvement of up to 43.5\% in PSNR over naive fine-tuning and surpassing joint training on some metrics.