Generative Modelling of BRDF Textures from Flash Images

TL;DR

This paper introduces a method to learn a latent space from flash images of natural materials, enabling the generation of diverse BRDF textures for realistic rendering, with promising results in visual matching and interpolation.

Contribution

It presents a novel approach combining convolutional encoding and neural networks to generate spatially varying BRDF parameters from flash images, improving material capture and rendering fidelity.

Findings

User study shows improved visual matching over previous methods.

Method produces diverse and consistent BRDF textures.

Enables realistic rendering in complex scenes.

Abstract

We learn a latent space for easy capture, consistent interpolation, and efficient reproduction of visual material appearance. When users provide a photo of a stationary natural material captured under flashlight illumination, first it is converted into a latent material code. Then, in the second step, conditioned on the material code, our method produces an infinite and diverse spatial field of BRDF model parameters (diffuse albedo, normals, roughness, specular albedo) that subsequently allows rendering in complex scenes and illuminations, matching the appearance of the input photograph. Technically, we jointly embed all flash images into a latent space using a convolutional encoder, and -- conditioned on these latent codes -- convert random spatial fields into fields of BRDF parameters using a convolutional neural network (CNN). We condition these BRDF parameters to match the visual characteristics (statistics and spectra of visual features) of the input under matching light. A user study compares our approach favorably to previous work, even those with access to BRDF supervision.