Sparse-DeRF: Deblurred Neural Radiance Fields from Sparse View

TL;DR

Sparse-DeRF introduces a novel regularization approach to construct deblurred neural radiance fields from sparse blurry views, effectively reducing overfitting and improving scene reconstruction quality in practical scenarios.

Contribution

It presents a new regularization framework for sparse-view DeRF, incorporating surface smoothness, modulated gradient scaling, and perceptual distillation to handle joint optimization challenges.

Findings

Effective from as few as 2 blurry views

Significant reduction in overfitting artifacts

Enhanced radiance field quality demonstrated

Abstract

Recent studies construct deblurred neural radiance fields~(DeRF) using dozens of blurry images, which are not practical scenarios if only a limited number of blurry images are available. This paper focuses on constructing DeRF from sparse-view for more pragmatic real-world scenarios. As observed in our experiments, establishing DeRF from sparse views proves to be a more challenging problem due to the inherent complexity arising from the simultaneous optimization of blur kernels and NeRF from sparse view. Sparse-DeRF successfully regularizes the complicated joint optimization, presenting alleviated overfitting artifacts and enhanced quality on radiance fields. The regularization consists of three key components: Surface smoothness, helps the model accurately predict the scene structure utilizing unseen and additional hidden rays derived from the blur kernel based on statistical tendencies of real-world; Modulated gradient scaling, helps the model adjust the amount of the backpropagated gradient according to the arrangements of scene objects; Perceptual distillation improves the perceptual quality by overcoming the ill-posed multi-view inconsistency of image deblurring and distilling the pre-deblurred information, compensating for the lack of clean information in blurry images. We demonstrate the effectiveness of the Sparse-DeRF with extensive quantitative and qualitative experimental results by training DeRF from 2-view, 4-view, and 6-view blurry images.