Strata-NeRF : Neural Radiance Fields for Stratified Scenes

TL;DR

Strata-NeRF introduces a neural radiance field model that effectively captures layered, multi-level scenes by conditioning on vector quantized representations, enabling seamless scene switching and high-fidelity view synthesis.

Contribution

It is the first to model stratified scenes with a single NeRF conditioned on VQ latent representations, improving multi-layer scene modeling and view synthesis.

Findings

Effectively captures multi-layered scenes with minimal artifacts.

Outperforms existing methods in synthetic and real-world datasets.

Achieves high-fidelity novel view synthesis across scene levels.

Abstract

Neural Radiance Field (NeRF) approaches learn the underlying 3D representation of a scene and generate photo-realistic novel views with high fidelity. However, most proposed settings concentrate on modelling a single object or a single level of a scene. However, in the real world, we may capture a scene at multiple levels, resulting in a layered capture. For example, tourists usually capture a monument's exterior structure before capturing the inner structure. Modelling such scenes in 3D with seamless switching between levels can drastically improve immersive experiences. However, most existing techniques struggle in modelling such scenes. We propose Strata-NeRF, a single neural radiance field that implicitly captures a scene with multiple levels. Strata-NeRF achieves this by conditioning the NeRFs on Vector Quantized (VQ) latent representations which allow sudden changes in scene structure. We evaluate the effectiveness of our approach in multi-layered synthetic dataset comprising diverse scenes and then further validate its generalization on the real-world RealEstate10K dataset. We find that Strata-NeRF effectively captures stratified scenes, minimizes artifacts, and synthesizes high-fidelity views compared to existing approaches.