Color-NeuS: Reconstructing Neural Implicit Surfaces with Color

TL;DR

Color-NeuS introduces a method for reconstructing textured 3D meshes from images by combining neural implicit surface reconstruction with a global color network, outperforming existing methods especially in occluded and variable lighting scenarios.

Contribution

It presents a novel approach that integrates mesh extraction with color reconstruction using neural implicit functions and a relighting network, enhancing multi-view object reconstruction.

Findings

Outperforms existing methods in occlusion and lighting variation scenarios.

Achieves high-quality mesh and color reconstruction on multiple datasets.

Demonstrates robustness across diverse real-world conditions.

Abstract

The reconstruction of object surfaces from multi-view images or monocular video is a fundamental issue in computer vision. However, much of the recent research concentrates on reconstructing geometry through implicit or explicit methods. In this paper, we shift our focus towards reconstructing mesh in conjunction with color. We remove the view-dependent color from neural volume rendering while retaining volume rendering performance through a relighting network. Mesh is extracted from the signed distance function (SDF) network for the surface, and color for each surface vertex is drawn from the global color network. To evaluate our approach, we conceived a in hand object scanning task featuring numerous occlusions and dramatic shifts in lighting conditions. We've gathered several videos for this task, and the results surpass those of any existing methods capable of reconstructing mesh alongside color. Additionally, our method's performance was assessed using public datasets, including DTU, BlendedMVS, and OmniObject3D. The results indicated that our method performs well across all these datasets. Project page: https://colmar-zlicheng.github.io/color_neus.