FNIN: A Fourier Neural Operator-based Numerical Integration Network for Surface-form-gradients

TL;DR

This paper introduces FNIN, a novel Fourier neural operator-based network for surface reconstruction from gradients, significantly improving accuracy and efficiency in high-resolution, complex scenarios.

Contribution

The paper presents a two-stage framework using Fourier neural operators and attention mechanisms for improved surface reconstruction from gradients, addressing discontinuities and high-resolution challenges.

Findings

Achieves errors of less than 0.1 mm on tested objects.

Outperforms state-of-the-art solvers in accuracy and efficiency.

Effectively handles high-resolution images with complex data.

Abstract

Surface-from-gradients (SfG) aims to recover a three-dimensional (3D) surface from its gradients. Traditional methods encounter significant challenges in achieving high accuracy and handling high-resolution inputs, particularly facing the complex nature of discontinuities and the inefficiencies associated with large-scale linear solvers. Although recent advances in deep learning, such as photometric stereo, have enhanced normal estimation accuracy, they do not fully address the intricacies of gradient-based surface reconstruction. To overcome these limitations, we propose a Fourier neural operator-based Numerical Integration Network (FNIN) within a two-stage optimization framework. In the first stage, our approach employs an iterative architecture for numerical integration, harnessing an advanced Fourier neural operator to approximate the solution operator in Fourier space. Additionally, a self-learning attention mechanism is incorporated to effectively detect and handle discontinuities. In the second stage, we refine the surface reconstruction by formulating a weighted least squares problem, addressing the identified discontinuities rationally. Extensive experiments demonstrate that our method achieves significant improvements in both accuracy and efficiency compared to current state-of-the-art solvers. This is particularly evident in handling high-resolution images with complex data, achieving errors of fewer than 0.1 mm on tested objects.