StructNeRF: Neural Radiance Fields for Indoor Scenes with Structural Hints

TL;DR

StructNeRF enhances neural radiance fields for indoor scene view synthesis by leveraging structural hints and self-supervised depth constraints, significantly improving quality with sparse inputs.

Contribution

It introduces a novel approach that uses structural cues and depth constraints to improve NeRF's performance on sparse-view indoor scenes.

Findings

Outperforms state-of-the-art methods quantitatively.

Achieves superior qualitative view synthesis results.

Effectively handles textured and non-textured regions.

Abstract

Neural Radiance Fields (NeRF) achieve photo-realistic view synthesis with densely captured input images. However, the geometry of NeRF is extremely under-constrained given sparse views, resulting in significant degradation of novel view synthesis quality. Inspired by self-supervised depth estimation methods, we propose StructNeRF, a solution to novel view synthesis for indoor scenes with sparse inputs. StructNeRF leverages the structural hints naturally embedded in multi-view inputs to handle the unconstrained geometry issue in NeRF. Specifically, it tackles the texture and non-texture regions respectively: a patch-based multi-view consistent photometric loss is proposed to constrain the geometry of textured regions; for non-textured ones, we explicitly restrict them to be 3D consistent planes. Through the dense self-supervised depth constraints, our method improves both the geometry and the view synthesis performance of NeRF without any additional training on external data. Extensive experiments on several real-world datasets demonstrate that StructNeRF surpasses state-of-the-art methods for indoor scenes with sparse inputs both quantitatively and qualitatively.