Fine-scale Surface Normal Estimation using a Single NIR Image

TL;DR

This paper introduces a method for estimating fine-scale surface normals from a single NIR image using a generative adversarial network, incorporating physical constraints for improved accuracy and generality across different datasets.

Contribution

It proposes a novel approach combining GANs with physical constraints for accurate surface normal estimation from single NIR images, even with uncalibrated lighting.

Findings

Effective in recovering sharp surface normals

Generalizes well across different datasets

Outperforms existing methods in accuracy

Abstract

We present surface normal estimation using a single near infrared (NIR) image. We are focusing on fine-scale surface geometry captured with an uncalibrated light source. To tackle this ill-posed problem, we adopt a generative adversarial network which is effective in recovering a sharp output, which is also essential for fine-scale surface normal estimation. We incorporate angular error and integrability constraint into the objective function of the network to make estimated normals physically meaningful. We train and validate our network on a recent NIR dataset, and also evaluate the generality of our trained model by using new external datasets which are captured with a different camera under different environment.