J-NeuS: Joint field optimization for Neural Surface reconstruction in urban scenes with limited image overlap

TL;DR

J-NeuS is a hybrid neural surface reconstruction method that effectively handles limited image overlap in urban scenes, achieving superior accuracy in reconstructing large areas and fine details from driving sequences.

Contribution

It introduces a joint optimization approach with cross-representation uncertainty estimation for improved urban scene reconstruction.

Findings

Outperforms state-of-the-art methods on major driving datasets.

Accurately reconstructs large urban areas with limited image overlap.

Effectively captures fine structures in complex environments.

Abstract

Reconstructing the surrounding surface geometry from recorded driving sequences poses a significant challenge due to the limited image overlap and complex topology of urban environments. SoTA neural implicit surface reconstruction methods often struggle in such setting, either failing due to small vision overlap or exhibiting suboptimal performance in accurately reconstructing both the surface and fine structures. To address these limitations, we introduce J-NeuS, a novel hybrid implicit surface reconstruction method for large driving sequences with outward facing camera poses. J-NeuS cross-representation uncertainty estimation to tackle ambiguous geometry caused by limited observations. Our method performs joint optimization of two radiance fields in addition to guided sampling achieving accurate reconstruction of large areas along with fine structures in complex urban scenarios. Extensive evaluation on major driving datasets demonstrates the superiority of our approach in reconstructing large driving sequences with limited image overlap, outperforming concurrent SoTA methods.