ImmersiveNeRF: Hybrid Radiance Fields for Unbounded Immersive Light Field Reconstruction

TL;DR

This paper introduces a hybrid radiance field approach for unbounded immersive light field reconstruction, enabling high-quality rendering and view extrapolation by separating foreground and background with adaptive strategies.

Contribution

It proposes a novel hybrid radiance field model with adaptive sampling and segmentation regularization, along with a new large-scale immersive light field dataset for improved AR/VR applications.

Findings

Effective separation of foreground and background improves rendering quality.

Adaptive sampling enhances convergence and segmentation clarity.

The method outperforms existing techniques in unbounded light field reconstruction.

Abstract

This paper proposes a hybrid radiance field representation for unbounded immersive light field reconstruction which supports high-quality rendering and aggressive view extrapolation. The key idea is to first formally separate the foreground and the background and then adaptively balance learning of them during the training process. To fulfill this goal, we represent the foreground and background as two separate radiance fields with two different spatial mapping strategies. We further propose an adaptive sampling strategy and a segmentation regularizer for more clear segmentation and robust convergence. Finally, we contribute a novel immersive light field dataset, named THUImmersive, with the potential to achieve much larger space 6DoF immersive rendering effects compared with existing datasets, by capturing multiple neighboring viewpoints for the same scene, to stimulate the research and AR/VR applications in the immersive light field domain. Extensive experiments demonstrate the strong performance of our method for unbounded immersive light field reconstruction.