psPRF:Pansharpening Planar Neural Radiance Field for Generalized 3D Reconstruction Satellite Imagery

TL;DR

This paper introduces psPRF, a neural radiance field model for satellite imagery that generalizes across scenes and integrates pan-sharpening, improving 3D reconstruction from paired low-res RGB and high-res panchromatic images.

Contribution

The paper presents psPRF, a novel planar neural radiance field architecture with spectral-to-spatial convolution and projection loss for enhanced generalization and multimodal representation in satellite 3D reconstruction.

Findings

Achieves state-of-the-art performance on multi-scene satellite data.

Effectively generalizes across different satellite scenes.

Enhances multimodal representation with spectral-to-spatial convolution.

Abstract

Most current NeRF variants for satellites are designed for one specific scene and fall short of generalization to new geometry. Additionally, the RGB images require pan-sharpening as an independent preprocessing step. This paper introduces psPRF, a Planar Neural Radiance Field designed for paired low-resolution RGB (LR-RGB) and high-resolution panchromatic (HR-PAN) images from satellite sensors with Rational Polynomial Cameras (RPC). To capture the cross-modal prior from both of the LR-RGB and HR-PAN images, for the Unet-shaped architecture, we adapt the encoder with explicit spectral-to-spatial convolution (SSConv) to enhance the multimodal representation ability. To support the generalization ability of psRPF across scenes, we adopt projection loss to ensure strong geometry self-supervision. The proposed method is evaluated with the multi-scene WorldView-3 LR-RGB and HR-PAN pairs, and achieves state-of-the-art performance.