Surface Normal Clustering for Implicit Representation of Manhattan Scenes

TL;DR

This paper introduces a method leveraging the Manhattan scene prior to self-supervise surface normal clustering in implicit neural representations, enhancing novel view synthesis and 3D modeling without dense labels.

Contribution

It proposes a novel approach to use high-level Manhattan scene priors for self-supervision in implicit neural radiance fields, improving scene understanding without dense annotations.

Findings

Significant performance improvements over baselines.

Effective surface normal clustering using Manhattan priors.

Robustness across diverse indoor scenes.

Abstract

Novel view synthesis and 3D modeling using implicit neural field representation are shown to be very effective for calibrated multi-view cameras. Such representations are known to benefit from additional geometric and semantic supervision. Most existing methods that exploit additional supervision require dense pixel-wise labels or localized scene priors. These methods cannot benefit from high-level vague scene priors provided in terms of scenes' descriptions. In this work, we aim to leverage the geometric prior of Manhattan scenes to improve the implicit neural radiance field representations. More precisely, we assume that only the knowledge of the indoor scene (under investigation) being Manhattan is known -- with no additional information whatsoever -- with an unknown Manhattan coordinate frame. Such high-level prior is used to self-supervise the surface normals derived explicitly in the implicit neural fields. Our modeling allows us to cluster the derived normals and exploit their orthogonality constraints for self-supervision. Our exhaustive experiments on datasets of diverse indoor scenes demonstrate the significant benefit of the proposed method over the established baselines. The source code is available at https://github.com/nikola3794/normal-clustering-nerf.