Invertible Neural BRDF for Object Inverse Rendering

TL;DR

This paper presents a novel invertible neural network-based BRDF model and a Bayesian framework for joint reflectance and illumination estimation from a single image, advancing object inverse rendering techniques.

Contribution

The introduction of the invertible neural BRDF (iBRDF) model using normalizing flows and a Bayesian MAP framework for efficient joint reflectance and illumination estimation.

Findings

The iBRDF model accurately captures real-world reflectance data.

The proposed method effectively estimates reflectance and illumination from single images.

Experimental validation demonstrates improved inverse rendering performance.

Abstract

We introduce a novel neural network-based BRDF model and a Bayesian framework for object inverse rendering, i.e., joint estimation of reflectance and natural illumination from a single image of an object of known geometry. The BRDF is expressed with an invertible neural network, namely, normalizing flow, which provides the expressive power of a high-dimensional representation, computational simplicity of a compact analytical model, and physical plausibility of a real-world BRDF. We extract the latent space of real-world reflectance by conditioning this model, which directly results in a strong reflectance prior. We refer to this model as the invertible neural BRDF model (iBRDF). We also devise a deep illumination prior by leveraging the structural bias of deep neural networks. By integrating this novel BRDF model and reflectance and illumination priors in a MAP estimation formulation, we show that this joint estimation can be computed efficiently with stochastic gradient descent. We experimentally validate the accuracy of the invertible neural BRDF model on a large number of measured data and demonstrate its use in object inverse rendering on a number of synthetic and real images. The results show new ways in which deep neural networks can help solve challenging radiometric inverse problems.