Block-NeRF: Scalable Large Scene Neural View Synthesis

TL;DR

Block-NeRF enables scalable, large-scale scene rendering by decomposing city environments into trainable blocks, allowing for efficient updates and seamless scene integration, demonstrated on a neighborhood of San Francisco.

Contribution

It introduces a scene decomposition approach for NeRFs, making large-scale scene rendering scalable and adaptable with new architectural features for robustness and seamless integration.

Findings

Successfully rendered a neighborhood of San Francisco using 2.8 million images.

Decoupling scene size from rendering time enables arbitrarily large scene rendering.

Per-block updates and appearance alignment improve scene consistency and flexibility.

Abstract

We present Block-NeRF, a variant of Neural Radiance Fields that can represent large-scale environments. Specifically, we demonstrate that when scaling NeRF to render city-scale scenes spanning multiple blocks, it is vital to decompose the scene into individually trained NeRFs. This decomposition decouples rendering time from scene size, enables rendering to scale to arbitrarily large environments, and allows per-block updates of the environment. We adopt several architectural changes to make NeRF robust to data captured over months under different environmental conditions. We add appearance embeddings, learned pose refinement, and controllable exposure to each individual NeRF, and introduce a procedure for aligning appearance between adjacent NeRFs so that they can be seamlessly combined. We build a grid of Block-NeRFs from 2.8 million images to create the largest neural scene representation to date, capable of rendering an entire neighborhood of San Francisco.