Blind Recovery of Spatially Varying Reflectance from a Single Image

TL;DR

This paper introduces a novel single-image technique for estimating spatially varying materials on objects with unknown shape and lighting, using a parametric reflectance model and new priors.

Contribution

It presents a new method for joint inference of shape, lighting, and material properties from a single image, with a large dataset for training and evaluation.

Findings

Accurately recovers material parameters from images.

Predicts new renderings effectively using recovered parameters.

Low-order reflectance model fits many real-world materials.

Abstract

We propose a new technique for estimating spatially varying parametric materials from a single image of an object with unknown shape in unknown illumination. Our method uses a low-order parametric reflectance model, and incorporates strong assumptions about lighting and shape. We develop new priors about how materials mix over space, and jointly infer all of these properties from a single image. This produces a decomposition of an image which corresponds, in one sense, to microscopic features (material reflectance) and macroscopic features (weights defining the mixing properties of materials over space). We have built a large dataset of real objects rendered with different material models under different illumination fields for training and ground truth evaluation. Extensive experiments on both our synthetic dataset images as well as real images show that (a) our method recovers parameters with reasonable accuracy; (b) material parameters recovered by our method give accurate predictions of new renderings of the object; and (c) our low-order reflectance model still provides a good fit to many real-world reflectances.