Shadow Neural Radiance Fields for Multi-view Satellite Photogrammetry

TL;DR

This paper introduces Shadow Neural Radiance Fields (S-NeRF), a self-supervised method for shadow-aware 3D reconstruction and view synthesis of Earth observation scenes from satellite images, accounting for complex lighting conditions.

Contribution

S-NeRF extends neural radiance fields to model both direct and indirect illumination in satellite imagery without explicit shape supervision, improving shadow and color accuracy.

Findings

Reduces altitude and color errors in shaded areas.

Enables shadow detection and albedo synthesis.

Performs accurate 3D shape estimation from satellite images.

Abstract

We present a new generic method for shadow-aware multi-view satellite photogrammetry of Earth Observation scenes. Our proposed method, the Shadow Neural Radiance Field (S-NeRF) follows recent advances in implicit volumetric representation learning. For each scene, we train S-NeRF using very high spatial resolution optical images taken from known viewing angles. The learning requires no labels or shape priors: it is self-supervised by an image reconstruction loss. To accommodate for changing light source conditions both from a directional light source (the Sun) and a diffuse light source (the sky), we extend the NeRF approach in two ways. First, direct illumination from the Sun is modeled via a local light source visibility field. Second, indirect illumination from a diffuse light source is learned as a non-local color field as a function of the position of the Sun. Quantitatively, the combination of these factors reduces the altitude and color errors in shaded areas, compared to NeRF. The S-NeRF methodology not only performs novel view synthesis and full 3D shape estimation, it also enables shadow detection, albedo synthesis, and transient object filtering, without any explicit shape supervision.