GradientSurf: Gradient-Domain Neural Surface Reconstruction from RGB Video

TL;DR

GradientSurf is a real-time neural surface reconstruction method from RGB video that incrementally builds detailed 3D surfaces by solving gradient-domain equations, outperforming previous approaches especially in detailed regions.

Contribution

It introduces an online neural surface reconstruction algorithm that operates in the gradient domain, improving detail and fidelity over prior offline methods.

Findings

Reconstructs surfaces with more details in curved regions.

Achieves higher fidelity for small objects.

Operates in real-time from monocular RGB video.

Abstract

This paper proposes GradientSurf, a novel algorithm for real time surface reconstruction from monocular RGB video. Inspired by Poisson Surface Reconstruction, the proposed method builds on the tight coupling between surface, volume, and oriented point cloud and solves the reconstruction problem in gradient-domain. Unlike Poisson Surface Reconstruction which finds an offline solution to the Poisson equation by solving a linear system after the scanning process is finished, our method finds online solutions from partial scans with a neural network incrementally where the Poisson layer is designed to supervise both local and global reconstruction. The main challenge that existing methods suffer from when reconstructing from RGB signal is a lack of details in the reconstructed surface. We hypothesize this is due to the spectral bias of neural networks towards learning low frequency geometric features. To address this issue, the reconstruction problem is cast onto gradient domain, where zeroth-order and first-order energies are minimized. The zeroth-order term penalizes location of the surface. The first-order term penalizes the difference between the gradient of reconstructed implicit function and the vector field formulated from oriented point clouds sampled at adaptive local densities. For the task of indoor scene reconstruction, visual and quantitative experimental results show that the proposed method reconstructs surfaces with more details in curved regions and higher fidelity for small objects than previous methods.