VoxNeRF: Bridging Voxel Representation and Neural Radiance Fields for Enhanced Indoor View Synthesis

TL;DR

VoxNeRF introduces a voxel-guided sampling method and depth loss to improve indoor view synthesis, achieving higher quality and efficiency in real-time scenarios using geometry priors.

Contribution

It presents a novel voxel-guided sampling technique and depth loss that leverage geometry priors for faster, higher-quality indoor view synthesis.

Findings

Outperforms state-of-the-art methods on ScanNet datasets

Reduces training and rendering time significantly

Establishes new benchmarks for indoor view synthesis

Abstract

The generation of high-fidelity view synthesis is essential for robotic navigation and interaction but remains challenging, particularly in indoor environments and real-time scenarios. Existing techniques often require significant computational resources for both training and rendering, and they frequently result in suboptimal 3D representations due to insufficient geometric structuring. To address these limitations, we introduce VoxNeRF, a novel approach that utilizes easy-to-obtain geometry priors to enhance both the quality and efficiency of neural indoor reconstruction and novel view synthesis. We propose an efficient voxel-guided sampling technique that allocates computational resources selectively to the most relevant segments of rays based on a voxel-encoded geometry prior, significantly reducing training and rendering time. Additionally, we incorporate a robust depth loss to improve reconstruction and rendering quality in sparse view settings. Our approach is validated with extensive experiments on ScanNet and ScanNet++ where VoxNeRF outperforms existing state-of-the-art methods and establishes a new benchmark for indoor immersive interpolation and extrapolation settings.