InfoNorm: Mutual Information Shaping of Normals for Sparse-View Reconstruction

TL;DR

This paper introduces InfoNorm, a method that improves 3D surface reconstruction from limited multi-view images by explicitly encouraging mutual information among surface normals, reducing reliance on pre-trained models.

Contribution

The paper proposes a mutual information regularization technique for surface normals, enhancing neural surface reconstruction without heavily depending on pre-trained geometry models.

Findings

Improves surface reconstruction quality over state-of-the-art methods.

Effectively utilizes semantic and geometric features for correlated point identification.

Serves as a plugin for SDF-based neural surface representations.

Abstract

3D surface reconstruction from multi-view images is essential for scene understanding and interaction. However, complex indoor scenes pose challenges such as ambiguity due to limited observations. Recent implicit surface representations, such as Neural Radiance Fields (NeRFs) and signed distance functions (SDFs), employ various geometric priors to resolve the lack of observed information. Nevertheless, their performance heavily depends on the quality of the pre-trained geometry estimation models. To ease such dependence, we propose regularizing the geometric modeling by explicitly encouraging the mutual information among surface normals of highly correlated scene points. In this way, the geometry learning process is modulated by the second-order correlations from noisy (first-order) geometric priors, thus eliminating the bias due to poor generalization. Additionally, we introduce a simple yet effective scheme that utilizes semantic and geometric features to identify correlated points, enhancing their mutual information accordingly. The proposed technique can serve as a plugin for SDF-based neural surface representations. Our experiments demonstrate the effectiveness of the proposed in improving the surface reconstruction quality of major states of the arts. Our code is available at: \url{https://github.com/Muliphein/InfoNorm}.