RNb-NeuS: Reflectance and Normal-based Multi-View 3D Reconstruction

TL;DR

This paper presents RNb-NeuS, a novel 3D reconstruction method that integrates reflectance and normal maps via a joint re-parameterization, improving detail and accuracy over existing multi-view photometric stereo techniques.

Contribution

The paper introduces a unified neural volume rendering approach that combines reflectance and normal maps with a single optimization, outperforming prior methods in MVPS benchmarks.

Findings

Outperforms state-of-the-art MVPS methods in F-score, Chamfer distance, and angular error.

Enhances detailed reconstruction of high-curvature and low-visibility areas.

Achieves better integration of reflectance and normal information in 3D reconstruction.

Abstract

This paper introduces a versatile paradigm for integrating multi-view reflectance (optional) and normal maps acquired through photometric stereo. Our approach employs a pixel-wise joint re-parameterization of reflectance and normal, considering them as a vector of radiances rendered under simulated, varying illumination. This re-parameterization enables the seamless integration of reflectance and normal maps as input data in neural volume rendering-based 3D reconstruction while preserving a single optimization objective. In contrast, recent multi-view photometric stereo (MVPS) methods depend on multiple, potentially conflicting objectives. Despite its apparent simplicity, our proposed approach outperforms state-of-the-art approaches in MVPS benchmarks across F-score, Chamfer distance, and mean angular error metrics. Notably, it significantly improves the detailed 3D reconstruction of areas with high curvature or low visibility.