PlaNeRF: SVD Unsupervised 3D Plane Regularization for NeRF Large-Scale Scene Reconstruction

TL;DR

PlaNeRF introduces an SVD-based plane regularization technique that enhances NeRF's 3D geometry reconstruction from RGB images, outperforming existing methods in large-scale outdoor scene modeling and rendering quality.

Contribution

The paper presents a novel SVD-based plane regularization method that improves NeRF's geometry accuracy without relying on geometric priors, suitable for large-scale outdoor scenes.

Findings

Outperforms popular regularization methods in geometry accuracy.

Achieves state-of-the-art rendering quality on KITTI-360 benchmark.

Effectively reconstructs 3D structure in low-texture outdoor scenes.

Abstract

Neural Radiance Fields (NeRF) enable 3D scene reconstruction from 2D images and camera poses for Novel View Synthesis (NVS). Although NeRF can produce photorealistic results, it often suffers from overfitting to training views, leading to poor geometry reconstruction, especially in low-texture areas. This limitation restricts many important applications which require accurate geometry, such as extrapolated NVS, HD mapping and scene editing. To address this limitation, we propose a new method to improve NeRF's 3D structure using only RGB images and semantic maps. Our approach introduces a novel plane regularization based on Singular Value Decomposition (SVD), that does not rely on any geometric prior. In addition, we leverage the Structural Similarity Index Measure (SSIM) in our loss design to properly initialize the volumetric representation of NeRF. Quantitative and qualitative results show that our method outperforms popular regularization approaches in accurate geometry reconstruction for large-scale outdoor scenes and achieves SoTA rendering quality on the KITTI-360 NVS benchmark.