Multiplane Prior Guided Few-Shot Aerial Scene Rendering

TL;DR

This paper introduces MPNeRF, a novel method that leverages geometric regularities and a Multiplane Prior guided by a SwinV2 Transformer to improve few-shot aerial scene rendering with NeRF, significantly outperforming existing methods.

Contribution

The paper pioneers the use of Multiplane Prior guided NeRF for aerial scenes, integrating advanced image comprehension to enhance sparse view rendering.

Findings

Triples SSIM and LPIPS performance with only three views.

Outperforms state-of-the-art methods in non-aerial contexts.

Leverages aerial geometric regularities for improved NeRF training.

Abstract

Neural Radiance Fields (NeRF) have been successfully applied in various aerial scenes, yet they face challenges with sparse views due to limited supervision. The acquisition of dense aerial views is often prohibitive, as unmanned aerial vehicles (UAVs) may encounter constraints in perspective range and energy constraints. In this work, we introduce Multiplane Prior guided NeRF (MPNeRF), a novel approach tailored for few-shot aerial scene rendering-marking a pioneering effort in this domain. Our key insight is that the intrinsic geometric regularities specific to aerial imagery could be leveraged to enhance NeRF in sparse aerial scenes. By investigating NeRF's and Multiplane Image (MPI)'s behavior, we propose to guide the training process of NeRF with a Multiplane Prior. The proposed Multiplane Prior draws upon MPI's benefits and incorporates advanced image comprehension through a SwinV2 Transformer, pre-trained via SimMIM. Our extensive experiments demonstrate that MPNeRF outperforms existing state-of-the-art methods applied in non-aerial contexts, by tripling the performance in SSIM and LPIPS even with three views available. We hope our work offers insights into the development of NeRF-based applications in aerial scenes with limited data.